Categories
Kotlin ORM-Persistence

SQL and ORM alike Database Access with Kotlin Exposed.

Introduction

Almost all access to SQL-based databases is performed through the JDBC API. It provides a standardized way for Java programs to interact with databases, enabling seamless communication and data manipulation. It acts as an intermediary layer, shielding the application from the underlying complexities of different database systems.
It also makes use of the SQL language which is a standard to unify the interactions with the database, from inserting, updating, querying, and deleting data.

This SQL language, together with the JDBC API, abstracts away the differences in the database systems. It contains features to increase the performance like prepared statements which also help in securing your application by using parameters and not just ‘pasting’ values into the SQL query.

The API itself is rather old and still relies on a lot of manual actions and statements from the developer. Resources like result sets, statements, and connections need to be closed after we don’t need them anymore. This results in a lot of boilerplate code.

The SQL language itself is seen as problematic by a lot of developers. The SQL Language is completely different from Java itself. And thus not everyone is keen on learning something besides Java. Especially since there are frameworks these days that provide a solution that works completely within Java.

But the traditional ORM frameworks like Hibernate bring a lot of other challenges and problems as I have described in the previous blog – Why ORM and Data frameworks are not your best option. So be careful in what features and functionalities you use in your application.

In this blog, we look at the Exposed library written in Kotlin. It is a lightweight SQL library on top of the JDBC driver for the Kotlin language. Exposed has two flavors of database access: typesafe SQL wrapping DSL and lightweight Data Access Objects (DAO).

It is an alternative for Java-based solutions like Hibernate when you are working with Kotlin. Also, JetBrains recently announced that it will increase the efforts into the framework, see this blog.

Installation

Since the Exposed framework has 2 flavours, the dependencies that you should add are separated into different artifacts.

The core dependency contains all the basic code and functionality which is independent of the JDBC or DAO flavour. Depending on the flavour you want to use, you should add the dependency you need.

When using Gradle as the build system, you can add the following dependencies.

dependencies {
implementation "org.jetbrains.exposed:exposed-core:${exposed.version}"
implementation "org.jetbrains.exposed:exposed-dao:${exposed.version}"
implementation "org.jetbrains.exposed:exposed-jdbc:${exposed.version}"
}

In the case of Maven, the following snippet gives you all the information you need.

    <dependency>
        <groupId>org.jetbrains.exposed</groupId>
        <artifactId>exposed-core</artifactId>
        <version>${exposed.version}</version>
    </dependency>
    <dependency>
        <groupId>org.jetbrains.exposed</groupId>
        <artifactId>exposed-dao</artifactId>
        <version>${exposed.version}</version>
    </dependency>
    <dependency>
        <groupId>org.jetbrains.exposed</groupId>
        <artifactId>exposed-jdbc</artifactId>
        <version>${exposed.version}</version>
    </dependency>

Defining the DB connection, which can be based on a standard DataSource object or use Connection Pooling provided by a third-party library like HikariCP, just requires calling the org.jetbrains.exposed.sql.Database.Companion#connect() function. This will make sure that all parts of the framework will use the DB connection from this setting. You can create for example following

object DBSettings {

    private val config = HikariConfig().apply {
        jdbcUrl = "jdbc:postgresql://localhost:5432/postgres"
        driverClassName = "org.postgresql.ds.PGSimpleDataSource"
        username = "postgres"
        password = "mysecretpassword"
        maximumPoolSize = 10
    }

    val dataSource by lazy {
        HikariDataSource(config)
    }

    val db by lazy {
        Database.connect(dataSource)
    }
}

Where you obviously retrieve the connection parameters from an external location instead of just hardcoding them in the code as in the previous example.

Typesafe access

One of the challenges we have when working with databases is creating SQL statements and mapping them to instances. We can either execute a SQL query which is basically just a String and map each field by hand to our properties of the objects. This is can very tedious. Or we define some ‘entities’ like in ORM frameworks and let the framework generate the SQL query. But that can lead to unoptimised queries, especially when you need very complex queries.

The Exposed framework, like some other frameworks in Java and Kotlin, try to approach this in a very lightweight way that handles the mapping for you so that your code is more type-safe (is a certain database field used as a String or Number?) and name changes in database columns and code can be handled transparently.

Within the JDBC feature of Exposed, you can define Singletons in Kotlin that extend from the Table class and define the correspondence between the table and code. Due to the expressiveness of Kotlin, these definitions can be highly readable. For example, an instance that maps to a table ‘Users’ that has some fields and a foreign key to the table ‘Cities’ can be defined as

object Users : Table() {
    val id = integer("id").autoIncrement()  // Column<Int>
    val name = varchar("name", length = 50) // Column<String>
    val cityId = (integer("city_id").references(Cities.id)).nullable() // Column<Int?>

    override val primaryKey = PrimaryKey(id, name = "PK_User_ID") // name is optional here
}

Due to the ‘inheritance’ from Table, each singleton defined as in the example, gets a lot of functions to perform some actions.

Like inserting into the table, and retrieving the newly assigned id using the auto-increment functionality of the database on the fly.

    val amsterdamId = Cities.insert {
        it[name] = "Amsterdam"
    } get Cities.id

Or loop over each record in the table

    for (city in Cities.selectAll()) {
        println("${city[Cities.id]}: ${city[Cities.name]}")
    }

You can see that in each of these use cases, we only refer to the property of the singleton Users and thus the actual table field name is only defined once in the configuration of the table itself.

But the JDBC feature also allows to create more complex queries, for example

    (Users innerJoin Cities).slice(Users.name, Cities.name).select { Users.name.eq("Emma") }.forEach {
        println("${it[Users.name]} works in ${it[Cities.name]}")
    }

This joins the 2 tables, based on the information within the definition (look at .references(Cities.id) earlier on), selects the 2 name columns, adds a where clause on the user name, and prints out the information in this case.

So this JDBC feature of the Exposed framework solves a few important issues of working with the JDBC API, namely the mapping and the boilerplate code that is required when using the JDBC API purely.
Note however that the mapping of the result sets is done using a Map-like structure where the key represents the column name and the value the database field value. People who are used to working with ORM frameworks will find this a bit awkward but Kotlin gives you a highly readable syntax for it. Most of all, you as the developer are still in control of the queries that are generated and thus performance issues are less likely.

The DAO or ORM solution.

The DAO features of the Exposed Framework take a slightly different approach to the same problem space by making use of ‘Entities’. So you need to define your table mappings and how you run queries differently. This means that both features are not compatible, or at least, combining both approaches did not work quite well for me. They can be used in the same application, but the definition of the table is slightly different and an Entity object is also needed.

object Users : IntIdTable() {
    val name = varchar("name", length = 50) // Column<String>
    val city = reference("city", Cities)
}


class User(id: EntityID<Int>) : IntEntity(id) {
    companion object : IntEntityClass<User>(Users)

    var name by Users.name
    var city by City referencedOn Users.city
}

Here Users contains the definition of the table and assumes an autoincrement Primary Key for the field id. The User class defines an instance of a record where we even have an object for City and not just the foreign key value.

An insert statement can be written as

    val amsterdam = City.new {
        name = "Amsterdam"
    }

which results in an INSERT SQL statement at the moment we access the database again or at the end of the transaction. So it is not flushed to the database immediately.

Retrieving data is also done differently than in the JDBS feature case. An important fact that you need to know is that joins are executed lazily which might not be the optimal case for some of your queries. For example, when retrieving the record for ‘Emma’ and the city she works in, the retrieval of the City name is done separately and after the retrieval of the User record itself.

    User.find { Users.name.eq("Emma") }.forEach {
        println("${it.name} works in ${it.city.name}")
    }

As you see, here again, the inheritance brings some useful methods into the objects like the find method.

Of course, an eager-loading way of working is possible by explicitly defining the joins that need to be made. This again is a positive point for this framework concerning the eager and lazy loading issues you can have with the classic ORM tools like Hibernate. The next snippet shows the same query but now with joining the tables eagerly.

    // Eager loading
    val query = Users.innerJoin(Cities).select(
        Users.name.eq("Emma")
    )

    User.wrapRows(query).forEach {
        println("${it.name} works in ${it.city.name}")
    }

Code examples

You can find the complete example project at this location of my Kotlin Project on GitHub.

If you want to learn more abput the Exposed framework itself, this is the main page of the Wiki for the project.

Conclusion

The Exposed framework is solving the main issues of working with the JDBC API in a completely different way the other frameworks like Hibernate. The lightweight approach to avoid mapping and typesafe access is not unique, but due to the expressiveness of the Kotlin language, results in a DSL-like solution that is highly readable and understandable.

Because the JDBC feature of the framework, tries to stay as close as possible to the JDBC API itself, it makes use of a Map-like structure for the transferred data. If you like real objects for records or result set items, you can make use of the DAO functionality of the framework. This resembles the ORM frameworks but is much easier to use and more performant in the execution.

Training and Support

Interested in a training about efficient ORM tools or Hibernate usage in your project? In need of an expert to help you solve a problem, feel free to contact me.

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
ORM-Persistence performance

Why ORM and Data frameworks are not your best option

As you all know, the relational world of databases is quite different from the object-oriented one. Both can store the same data but they do it differently. Making it a challenge to retrieve and store it from an object-oriented language like Java to a SQL-based database.

In principle, frameworks like ORMs (Hibernate to name the most popular one) and ‘data’ frameworks, like Spring Data or the new Jakarta Data, manage to handle the conversation well. But only for simple cases.

This blog is mainly about the SQL case but also can be applied to the NoSQL frameworks. When you try to abstract away the underlying communication to the underlying system, you lose performance, functionality, and flexibility.

The JDBC connectivity

From the early days of Java, The Java Database Connectivity API made it possible to interact with the SQL databases. It allowed the Java developer to execute SQL statements and retrieve data for the application.

In those days, it was normal that every developer knew SQL very well and could write the most complex queries to retrieve the data required for the advanced use cases of their applications. Using joins, sub-queries, and grouping is not that hard and can easily be mastered within a week.

The challenge and difficulty is the usage of the JDBC API. In those early days, many APIs were designed so that the developer still needed to perform many actions. That leads to a lot of boilerplate code that is required to close statements, result sets, and connections. If you fail to do this properly, it results in resource leakage and failures in your application over time.

But there are other challenges. You need to manually assign each column value from a JDBC ResultSet to the properties of your objects. This is not the most rewarding piece of code that you as a developer write on a project.

Refactorings, where database fields change can also be a challenge since the SQLs you execute are actually just Strings, and thus at compile time of your application, they are not validated.

The benefits of the ORM

All the challenges described in the previous section, are addressed by the ORM tools like Hibernate. They simplify database interactions and bridge the gap between object-oriented programming languages and databases.

Firstly, ORM tools eliminate the need for developers to write repetitive and error-prone boilerplate code. That is hidden away in the framework code.

Another significant benefit of ORM tools is their ability to provide type-safe mappings between database tables and object-oriented classes. Initially defined through XML files and later on through the annotations available from Java 5 onwards, the tool handles the translation of data between the database and objects. Besides the mappings of single fields, it allows you to represent foreign key relations from the database. But also to represent concepts that do not exist in the database such as the OneToMany relation to have all children of each master record.

Although the SQL language is standardised, there are several versions like SQL-92, SQL-2003, and SQL-2023 where the JSON datatype is introduced. Not all databases support the same version and all databases use a custom, slightly different version. This is handled by the introduction of the ORM Query Language (Hibernate Query Language, JPA Query language, etc.) and a Dialect for each database that converts these queries to a format that is supported by the database.
This ORM Query Language is thus a subset of all possible features found in the databases and thus you cannot use the full power of the database.

Additionally, ORM tools offer a range of features such as support for database schema migrations, caching, and query optimization. They enable developers to work with databases more abstractly and intuitively, freeing them from having to think in terms of SQL statements and database-specific details. This abstraction allows for greater flexibility in choosing the underlying database system, but also in lower performance as it adds an additional rather complex layer and less functionality.

After the ORM tools were in use for several years, people saw that querying a table and filtering on one or a few fields, results in a handful of very similar statements.

This led to the creation of the data project like Spring Data. Instead of writing these few statements each time, these are derived from the method name of specially indicated interfaces.

List findByName(String name)

This kind of method just replaces 3 lines of code with the ORM tool.

The problems of the ORM

In the previous section, you can find many useful improvements in accessing the database by using an OM tool. But it has its own challenges and problems.

Probably the most common problem is the Lazy and Eager loading strategies and the ‘N+1 select issue’.
In almost all cases, you don’t need to retrieve the records of a table in isolation, but you also need to take into account the relations with other tables. This can be needed to have all the fields for filtering, or additional data for displaying on the screen.
Within the ORM tool, they are represented by the ManyToOne, OneToMany, or ManyToMany relation.

The ORM can decide to load the information of these related tables eagerly by including the table already in the query using a JOIN clause. Or after the main query is executed, issuing additional queries to load the details in the lazy case.

But this lazy case introduces the ‘N+1 select issue’ since after retrieving the results for the main query which has N rows, the ORM tool launches N queries to retrieve the detail collection of each row.

So is eagerly preferred over lazy loading then? No, not at all. Since many tables are connected, eagerly loading retrieves in most cases information from tables that are not needed, making queries complex and slow. You must decide on an individual basis if data is required or not.

Some real-world cases and best practices

In my 20+ years career as a Java Developer, I was called in on many projects that were already in production and experienced some issues or needed some advanced functionality.
In almost all cases, the issues could be retraced to how the ORM mapping and tool were used in the project.

I’ll briefly discuss some cases and explain the solutions that I applied to the problems.

One case is about the eager loading and the lack of a proper design for the Entity layer.
They called me in on a project where there was a performance problem on the main page of the application that showed some kind of overview, minimal dashboard, for the user. The page showed about 10 values, so not much information but it actually took about 45 seconds to load.

The reason was quickly found when I activated the SQL tracing to see what queries were sent to the database. To get the data for the main page, there were 1329 queries executed. The developer only issued 5 queries, but the development team used the lazy loading configuration and did not specify any FETCHING strategy on the queries themselves.

Since all tables are connected, which is the common case, some queries that touch many tables or collect data from some detail collections are not performant when relying on standard ORM tool behavior due to the lazy option.

In this case, the usage of eager loading would not solve the problem as the 5 queries would become very large, touching many tables within the database. These queries are also very slow, and thus not a solution.

The solution was actually very simple, create 5 ‘native queries’ that can retrieve the required values very efficiently from the database. The ORM tool can execute a native query. You still can make use of the default mapping, or retrieve a collection of values when the query is not returning entire table rows but only some values. But you bypass the ORM Query language conversion to SQL, so faster anyway, and you submit the ideal query to the database. Again the fastest option.

So, if you haven’t done it already, take that intermediate or advanced SQL course so that you become a pro in writing complex queries. Use native queries if you touch 3 or more tables as you can write them more efficiently than the ORM which is designed for simple cases. Use the Lazy fetching strategy but define in the query if you need the data or not by using a JOIN FETCH clause for your simple queries.

The second case I want to discuss is a project where they relied heavily on Spring Data. They used the method name convention to define the query or in several cases used the @Query annotation to instruct what should be executed.

During development, everything went smoothly but quickly some performance issues arose when running in production. The difference, in production there are not 10 or 20 records but 10,000 and more.

Since the development team used Spring Data, they included many eager relations and also used ToMany relations extensively to get the info they needed ‘automagically’. Since these options result in joining many tables, in many cases not needed for the situation, the queries became slow when using more records.

The solution in this case was again relying on native queries that could optimally retrieve the data, reduce the usage of Spring Data interface methods, and define query and fetch strategies within queries.

In general, avoid, or simply don’t use at all, the usage of the ToMany relations as they are not available ‘naturally’ in databases and require complex or additional queries.
Avoid the usage of the Data frameworks when querying more than 1 table or don’t use them at all since writing 3 statements is not a problem. Or is it?
Avoid the eager loading definition and specify the JOIN FETCH clause when you need the info.

This is of course a quick and limited overview. For example, did you know that you can’t use database pagination when you use JOIN FETCH? That you can’t use a sub-query in a JOIN in the ORM Query language?

Conclusion

The ORM tools solve a few important issues, the boilerplate code required with the JDBC API and the mapping of database field values to Java object properties. But it can introduce a lot of troubles in your projects regarding performance which is mostly only discovered when running in production with larger datasets.

So as a rule of thumb, do not use eager loading and use JOIN FETCH clauses when needed, do not use any ToMany relation as they make your queries complex, and write all your queries, other than the very simple one table ones, yourself using native queries.

Since we only should use a very limited set of functionality of the ORM tools like Hibernate, why don’t we drop it altogether and just use a tool that reduces the boilerplate code and solves the mapping issue (type-safe column names and values)? Tools like JOOQ or the Expose framework written in Kotlin, are 2 examples of tools that implement the best practices I described here without the overhead of an ORM.
Interested in an introduction to the Expose framework, I’ll give an overview in my next blog.

Training and support

Interested in a training about efficient ORM tools or Hibernate usage in your project? In need of an expert to help you solve a problem, feel free to contact me.

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
Jakarta EE ORM-Persistence

Why ORM and Data frameworks are not your best option

Subscribe to continue reading

Subscribe to get access to the rest of this post and other subscriber-only content.

Already a subscriber?

Categories
Jakarta EE Ktor Security

Comparing JWT Token Usage in Spring Boot, Quarkus, Jakarta, and Kotlin Ktor: A Framework Exploration – Part 4

Since this topic became very extensive, I decided to split up the blog into 4 parts. To keep blog lengths manageable. Here is the split up

Part 1: Introduction
Part 2: Payara, Spring Boot and Quarkus
Part 3: Ktor and Atbash Runtime
Part 4: Discussion and conclusion (this one)

For an introduction around JWT Tokens, you can have a look at the first part of this blog. It also contains a description how the Keycloak service is created for the example programs described in this part.
Part 2 and 3 contains the description of the example application for each runtime.

Discussion

In parts 2 and 3, I showed the most important aspects of using a JWT token with Payara Micro (Jakarta EE), Spring Boot, Quarkus, Kotlin, and Atbash Runtime. The JWT tokens themselves are standardised but how you must use them in the different runtimes is not defined and thus different. Although there exists the MicroProfile JWT Auth specification, even those runtimes that follow it, have differences in how it should be activated and how roles should be verified, especially when you don’t want to check a role. The specification, besides duplicating a few things from the JWT specification itself like how validation needs to be done, only defines how a MicroProfile application should retrieve claim values.

It is obvious that for each runtime we need to add some dependency that brings in the code to handle the JWT tokens. But for several of these runtimes, you also need to activate the functionality. This is the case for Payara Micro through the @LoginConfig and also for Atbash Runtime since the functionality is provided there by a non-core module.

Another configuration aspect is the definition of the location of the certificates. Spring Boot is the only one that makes use of the OAuth2 / OpenId Connect well know endpoint for this. The other runtimes require you to specify the URL where the keys can be retrieved in a certain format. This allows for more flexibility of course and potential support for providers that do not follow the standard in all its extends. But since we are talking about security, it would probably be better that only those certified, properly tested providers would be used as is the case with the Spring Boot implementation.

The main difference in using a JWT token on runtime is how the roles are verified. Not only is not specified which claim should hold the role names, nor is it defined how the authorization should be performed. This leads to important differences between the runtimes.

Within Kotlin Ktor, We should define a security protocol for each different role we want to check and assign it a name. Or you create a custom extension function that allows you to specify the role at the endpoint as I have done in the example. But important to note is that we need to be explicit in each case. Which role or if no role at all is required, we need to indicate this.

This is not the case for the other runtimes, except the Atbash Runtime.

When you don’t use any annotation on the JAX-RS method with Payara Micro and Spring Boot, no role is required, only a valid JWT token. But with Quarkus, when not specifying anything, the endpoint becomes publicly accessible. This is not a good practice because when you as a developer forget to put an annotation, the endpoint becomes available for everyone, or at least any authenticated user for certain runtime. This violates the “principle of least privilege” that by default, a user has no rights and you explicitly need to define who is allowed to call that action. That is the reason why Atbash Runtime treats the omission of an annotation to check on roles as an error and hides the endpoint and shows a warning in the log.

If you do not want to check for a role when using Atbash Runtime, you can annotate the JAX-RS method with @PermitAll. The JavaDoc says “Specifies that all security roles are allowed to invoke the specified method(s)” and thus it is clearly about the authorization on the endpoint. But if you use @PermitAll in Payara Micro, the endpoint becomes publicly accessible, dropping also authentication. That is not the intention of the annotation if you ask me. Although the Javadoc might be to blame for this as it mentions “that the specified method(s) are ‘unchecked'” which might be interpreted as no check at all.

Conclusion

All major frameworks and runtimes have support for using JWT Tokens within your application to authenticate and authorise a client call to a JAX-RS endpoint. When adding the necessary dependency to have the code available and adding some minimal configuration like defining where the keys can be retrieved to verify the signature, you are ready to go. The only exception here might be Kotlin Ktor where you are confronted with a few manual statements about the verification and validation of the token. It is not completely hidden away.

The most important difference lies in how the check for the roles is done. And especially in the case that we don’t require any role, just a valid JWT token. Only Atbash Runtime applies the “principle of least privilege”. On the other runtimes, forgetting to define a check for a role leads to the fact that the endpoint becomes accessible to any authenticated user or even worse, publicly accessible.

There is also confusion around @PermitAll which according to the java doc is about authorization, but in Jakarta EE runtime like Payara Micro, the endpoint also suddenly becomes publicly accessible.

Interested in running an example on the mentioned runtimes, check out the directories in the https://github.com/rdebusscher/Project_FF/tree/main/jwt repo which work with KeyCloak as the provider.

Training and Support

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
Jakarta EE Ktor Security

Comparing JWT Token Usage in Spring Boot, Quarkus, Jakarta, and Kotlin Ktor: A Framework Exploration – Part 3

Since this topic became very extensive, I decided to split up the blog into 4 parts. To keep blog lengths manageable. Here is the split up

Part 1: Introduction
Part 2: Payara, Spring Boot and Quarkus
Part 3: Ktor and Atbash Runtime (this one)
Part 4: Discussion and conclusion

For an introduction around JWT Tokens, you can have a look at the first part of this blog. It also contains a description how the Keycloak service is created for the example programs described in this part.
Part 2 contains the description for Payara Micro, Spring Boot and Quarkus.

Ktor

Also within Ktor there is some excellent support for using JWT tokens although we need to code a little bit more if we want to have support for rotating public keys and easy checks on the roles within the tokens.

But first, Let us start again with the dependencies you need within your application.

        <!-- Ktor authentication -->
        <dependency>
            <groupId>io.ktor</groupId>
            <artifactId>ktor-server-auth-jvm</artifactId>
            <version>${ktor_version}</version>
        </dependency>
        <!-- Ktor support for JWT -->
        <dependency>
            <groupId>io.ktor</groupId>
            <artifactId>ktor-server-auth-jwt-jvm</artifactId>
            <version>${ktor_version}</version>
        </dependency>

We need a dependency to add the authentication support and another one for having the JWT token as the source for authentication and authorisation.

Just as with the Payara and Quarkus case, we need to define the location to retrieve the public key, expected issuer, and audience through the configuration of our application. In our example application, this is provided in the application.yml file.

jwt:
  issuer: "http://localhost:8888/auth/realms/atbash_project_ff"
  audience: "account"

We programmatically read these values in our own code, so the keys can be whatever you like, they are not predetermined as with the other runtimes. In the example, you see that we also don’t define the location of the public key endpoint as we can derive that from the issuer value in the case of KeyCloak. But you are free to specify a specific URL for this value of course.

Configuration of the modules in Ktor is commonly done by creating an extension function on Application object, as I have also done in this example. This is the general structure of this function

fun Application.configureSecurity() {

    authentication {
        jwt("jwt-auth") {
            realm = "Atbash project FF"
            // this@configureSecurity refers to Application.configureSecurity()
            val issuer = this@configureSecurity.environment.config.property("jwt.issuer").getString()
            val expectedAudience = this@configureSecurity.environment.config.property("jwt.audience").getString()
            val jwkUrl = URL("$issuer/protocol/openid-connect/certs")
            val jwkProvider = UrlJwkProvider(jwkUrl)

            verifier {
        // not shown for brevity              

            }

            validate { credential ->
                // If we need validation of the roles, use authorizeWithRoles
                // We cannot define the roles that we need to be able to check this here.
                JWTPrincipal(credential.payload)
            }

            challenge { defaultScheme, realm ->
                // Response when verification fails
                // Ideally should be a JSON payload that we sent back
                call.respond(HttpStatusCode.Unauthorized, "$realm: Token is not valid or has expired")
            }
        }
    }

}

The function jwt("jwt-auth") { indicates that we define an authentication protocol based on the JWT tokens and we name it jwt-auth. We can name it differently and can have even multiple protocols in the same application as long ask we correctly indicate which protocol name we want at the endpoint.

The JWT protocol in Ktor requires 3 parts, a verification part, a validation one, and lastly how the challenge is handled.

The verification part defines how the verification of the token is performed and will be discussed in more detail in a moment. We can do further validation on the token by looking at the roles that are in the token. If you have many different roles, this leads to many different named JWT protocols. Therefore I opted in this example to write another extension function on the Route object that handles this requirement more generically. And the challenge part is executed to formulate a response for the client in case the validation of the token failed.

The verifier method defines how the verification of the token is performed. We make use of the UrlJwkProvider which can read the keys in the JWKS format which contains keys in a JSON format. But it doesn’t try to reread the endpoint in case the key is not found. This also means we cannot apply rotating keys for signing the JWT tokens which is recommended in production. Therefore, we make use of a small helper which caches the keys but read the endpoint again when the key is not found. This functionality could be improved to avoid a DOS attack by calling your endpoint with some random key ids which would put Keycloak or the JWT Token provider under stress.

            val jwkProvider = UrlJwkProvider(jwkUrl)

            verifier {
                val publicKey = PublicKeyCache.getPublicKey(jwkProvider, it)

                JWT.require(Algorithm.RSA256(publicKey, null))
                    .withAudience(expectedAudience)
                    .withIssuer(issuer)
                    .build()

            }

The other improvement that you can find in the example is the validation part. Since you only have the credential as input for this validation, you can check if the token has a certain role, but you can’t make this check dynamic based on the endpoint. As mentioned, this would mean that for each role that you want to check, you should make a different JWT check.

The example contains an extension function on the Route object so that you can define the role that you expect. This is how you can use this new authorizeWithRoles function

        authorizeWithRoles("jwt-auth", listOf("administrator")) {
            get("/protected/admin") {
                call.respondText("Protected Resource; Administrator Only ")
            }
        }

So besides the name for the protocol we like to use, you can also define a set of roles that you expect to be in the token. The function itself is not that long but a little complex because we add a new interceptor in the pipeline used by Ktor to handle the request. If you want to look at the details, have a look at the example code.

If you just need a valid token, without any check on the roles, you can make use of the standard Ktor functionality

        authenticate("jwt-auth") {
            get("/protected/user") {
                val principal = call.authentication.principal<JWTPrincipal>()
                //val username = principal?.payload?.getClaim("username")?.asString()
                val username = principal?.payload?.getClaim("preferred_username")?.asString()
                call.respondText("Hello, $username!")
            }
        }

This last snippet also shows how you can get access to the claims within the token. You can access the principal associated with- the request by requesting call.authentication.principal<JWTPrincipal>() where you immediately make the cast to the JWTPrincipal class. This contains the entire token content easily accessible from within your Kotlin code as you can see in the example where I retrieve the preferred_username.

You can review all code presented here in the example https://github.com/rdebusscher/Project_FF/tree/main/jwt/ktor.

Atbash Runtime

Atbash Runtime is a small modular Jakarta EE Core profile runtime. So by default, it doesn’t has support for using JWT tokens. But since these tokens are the de facto standard, there is an Atbash Runtime module that supports them so that you can use it for your application.

As a dependency, you can add this JWT supporting module to your project

        <dependency>
            <!-- Adds JWT Support in the case we are using the Jakarta Runner, no addition of the MP JWT Auth API required -->
            <!-- Otherwise, when not using Jakarta Runner, the addition of JWT Auth API as provided is enough if you are using Atbash Runner Jar executable -->
            <groupId>be.atbash.runtime</groupId>
            <artifactId>jwt-auth-module</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>

Since we use the Jakarta Runner feature of the Atbash runtime, which allows you to execute your web application through a simple main method, we need to add the module itself. If you run your application as a war file, make sure you activate the JWT module within the configuration so that the module is active.

The JWT support within Atbash runtime is also based on the Microprofile JWT Auth specification, so you will see many similarities with the Payara and Quarkus examples we have discussed in part 2 of this blog.

Configuration requires the 3 values for public key location, expected issuer, and audience.

mp.jwt.verify.publickey.location=http://localhost:8888/auth/realms/atbash_project_ff/protocol/openid-connect/certs
mp.jwt.verify.issuer=http://localhost:8888/auth/realms/atbash_project_ff
mp.jwt.verify.audiences=account

You are also required to indicate the @LoginConfig (in case you are executing your application as a WAR file) so that the JWT Module is active for the application. But there is no need to define @DeclareRoles as Atbash Runtime takes the value of the individual @RolesAllowed as valid roles.

A difference with Payara, for example, is that you need to add @PermitAll to a method when you don’t want to check on any roles. Within Atbash Runtime there is the “principle of least privilege” implemented. If you don’t specify anything on a JAX_RS method no client can call it. This is to avoid that you forget to define some security requirements and expose the endpoint without any checks. The JavaDoc says “Specifies that all security roles are allowed to invoke the specified method(s)” and thus it is clearly what we need. Although, some runtimes, including Payara, interpret this differently and I’ll go deeper on this topic in part 4.

The example code is located at https://github.com/rdebusscher/Project_FF/tree/main/jwt/atbash.

Discussion

In the last part of the blog, I’ll have a discussion about similarities and differences. These differences are especially important when you don’t want to have a check on a role within the token.

Part 1: Introduction
Part 2: Payara, Spring Boot and Quarkus
Part 3: Ktor and Atbash Runtime (this one)
Part 4: Discussion and conclusion

Training and Support

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
Jakarta EE Ktor Security

Comparing JWT Token Usage in Spring Boot, Quarkus, Jakarta, and Kotlin Ktor: A Framework Exploration – Part 2

Since this topic became very extensive, I decided to split up the blog into 4 parts. To keep blog lengths manageable. Here is the split up

Part 1: Introduction
Part 2: Payara, Spring Boot and Quarkus (this one)
Part 3: Ktor and Atbash Runtime
Part 4: Discussion and conclusion

For an introduction around JWT Tokens, you can have a look at the first part of this blog. It also contains a description how the Keycloak service is created for the example programs described in this part.

Payara

As an example of how you can work with JWT tokens with Jakarta EE and MicroProfile, we make use of Payara Micro.

The JWT Token support is provided by MicroProfile, so add the dependency to your project.

        <dependency>
            <groupId>org.eclipse.microprofile</groupId>
            <artifactId>microprofile</artifactId>
            <version>6.0</version>
            <type>pom</type>
            <scope>provided</scope>
        </dependency>

We are using MicroProfile 6, which requires Jakarta EE 10 runtime as this is the version that is supported by the Payara Micro community edition.

As configuration, we need to provide the endpoint where the MicroProfile JWT Auth implementation can retrieve the public key that is required to validate the content of the token against the provided signature. This can be done by specifying mp.jwt.verify.publickey.location configuration key. Two other configuration keys are required, one that verifies if the issuer of the token is as expected and the audience claim is the other one.

Other configuration aspects are the indication that a JWT token will be used as authentication and authorization for the endpoints through the @LoginConfig annotation. The @DeclareRoles annotation is a Jakarta EE annotation that indicates which roles are recognised and can be used. These annotations can be placed on any CDI bean.

@LoginConfig(authMethod = "MP-JWT")
@DeclareRoles({"administrator"})

On the JAX-RS method, we can add the @RolesAllowed annotation to indicate the role that must be present in the token before the client is allowed to call the endpoint.

    @GET
    @Path("/admin")
    @RolesAllowed("administrator")
    public String getAdminMessage() {

When there is no annotation placed on the method, only a valid JWT token is required to call the endpoint. Also, have a look at the part 4 of this blog for some important info and differences between runtimes.

Through the MicroProfile JWT Auth specification, we can also access one or all the claims that are present in the token. The following snippet shows how you can access a single claim or the entire token in a CDI bean or JAX-RS resource class.

    @Inject
    @Claim("preferred_username")
    private String name;

    @Inject
    private JsonWebToken jsonWebToken;
    // When you need access to every aspect of the JWT token.

The entire example can be found in the project https://github.com/rdebusscher/Project_FF/tree/main/jwt/payara.

Spring Boot

Also, Spring Boot has excellent support for using JWT tokens for the authentication and authorization of rest endpoints. Besides the Spring Boot Security starter, the Oauth2 Resource Server dependency is required within your application. So you don’t need to handle the JWT token yourself in a programmatic way as some resources on the internet claim.

In our example, we use Spring Boot 3 and JDK 17.

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
    </dependency>

In contrast to MicroProfile where you need to provide several configuration keys, Spring boot makes use of the OpenId Connect specification where it is defined that the endpoint .well-known/openid-configuration provides all info. This includes the location of the public key required for the validation of the token against the signature and the value of the issuer. The location can be specified through a Spring Configuration resource.

spring.security.oauth2.resourceserver.jwt.issuer-uri=http://localhost:8888/auth/realms/atbash_project_ff
spring.security.oauth2.resourceserver.jwt.audiences=account

The audience value is not required to be defined, Spring Boot works without it. But it is a recommended configuration aspect to make sure that tokens are correctly used, especially when you use tokens for multiple applications.

You can either define the requirements for the roles that should be present in the token using a Spring Bean that extends the WebSecurityConfigurerAdapter class and the HttpSecurity builder, but I prefer the method-based approach.
With this approach, you can define the required role using the @PreAuthorize annotation

    @GetMapping("/admin")
    @PreAuthorize("hasAuthority('administrator')")
    public String getAdminMessage() {

It makes it easier to find out which role is required before a client can call the endpoint and also easier to verify if you didn’t make any error in the security configuration of your application. This method-based approach requires a small activation and mapping between the roles within the token and the authority we check in the annotation.

@Configuration
@EnableMethodSecurity
public class MethodSecurityConfig {
}

The configuration for the JWT token roles is provided by a JwtAuthenticationConverter bean.

    @Bean
    public JwtAuthenticationConverter jwtAuthenticationConverter() {
        JwtGrantedAuthoritiesConverter grantedAuthoritiesConverter = new JwtGrantedAuthoritiesConverter();
        grantedAuthoritiesConverter.setAuthorityPrefix("");
        grantedAuthoritiesConverter.setAuthoritiesClaimName("groups");

        JwtAuthenticationConverter jwtAuthenticationConverter = new JwtAuthenticationConverter();
        jwtAuthenticationConverter.setJwtGrantedAuthoritiesConverter(grantedAuthoritiesConverter);
        return jwtAuthenticationConverter;
    }

Within the REST methods, we can have access to the JWT token claims, just as with the Jakarta EE and MicroProfile example. We need to add a JwtAuthenticationToken parameter to the method which allows access to claims through the getTokenAttributes() method.

    @GetMapping("/user")
    public String getUser(JwtAuthenticationToken authentication) {
        Object username = authentication.getTokenAttributes().get("preferred_username");

The entire example can be found in the project https://github.com/rdebusscher/Project_FF/tree/main/jwt/spring.

Quarkus

The Quarkus support is also based on MicroProfile, so you will see several similarities with the Payara case I described earlier. The Quarkus example is based on the recent Quarkus 3.x version. As a dependency, we need two artifacts related to the JWT support provided by the SmallRye project. Although it seems you do not need the build one at first sight, as it is about creating JWT tokens within your application, the example did not work without it.

        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-jwt</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-jwt-build</artifactId>
        </dependency>

Since the SmallRye JWT implementation is also using the SMicroProfile JWT auth specification, the configuration through key-value pairs is identical to the Payara one. We need to define the location of the publicKey, and the expected values for the issuer and audience. In the example, I have defined them in the application.properties file, a Quarkus-specific configuration source. But as long as they can be retrieved through any of the supported configuration sources, it is ok.

Since Quarkus is not a Jakarta-compliant runtime, it doesn’t require any indication that the application will make use of the JWT tokens for authentication and authorisation. The existence of the two dependencies we added earlier to the project is enough. In this case, it is similar to the Spring Boot case where we also did not do this.

On the JAX-RS resource methods, we can indicate if we need a certain role within the token, or that just the token itself is required and no specific role is required. If a role is required, we can make use of the same @RolesAllowed annotation we encountered in the Payara example or we need to add the @Authenticated annotation if we just need a valid token.

    @GET
    @Path("/admin")
    @RolesAllowed("administrator")
    public String getAdminMessage() {
        return "Protected Resource; Administrator Only ";
    }

    @GET
    @Path("/user")
    @Authenticated
    // No roles specified, so only valid JWT is required
    public String getUser() {
        return "Protected Resource; user : " + name;
    }

This @Authenticated annotation is defined in the Quarkus Security artifact, brought in transitively, and indicates that an authenticated user is required. Without this annotation, the endpoint would become publicly accessible, without the need for any token or authentication method.

More on that in a part 4 of this blog.

The retrieval of the claims is again identical to the Payara case. The example project can be found at https://github.com/rdebusscher/Project_FF/tree/main/jwt/quarkus.

Runtimes

The Ktor and Atbash Runtime versions of the example application are described in part 3.

Part 1: Introduction
Part 2: Payara, Spring Boot and Quarkus (this one)
Part 3: Ktor and Atbash Runtime
Part 4: Discussion and conclusion

Training and Support

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
Jakarta EE Ktor Security

Comparing JWT Token Usage in Spring Boot, Quarkus, Jakarta, and Kotlin Ktor: A Framework Exploration – Part 1

Since this topic became very extensive, I decided to split up the blog into 4 parts. To keep blog lengths manageable. Here is the split up

Part 1: Introduction (this one)
Part 2: Payara, Spring Boot and Quarkus
Part 3: Ktor and Atbash Runtime
Part 4: Discussion and conclusion

But don’t worry, all these 4 parts will be released within the same week so that those people that are eager to process it in one go, do not need to wait a long time before the series is published.

Introduction

As the demand for secure and efficient authentication and authorization mechanisms grows, JSON Web Tokens (JWT) have emerged as a favored choice for developers. JWT tokens provide a modern approach to verifying user identity and defining access privileges within web applications. In this blog post, we will delve into the usage of JWT tokens across various frameworks, namely Spring Boot, Quarkus, Jakarta, and Kotlin Ktor. By comparing their implementation approaches, we aim to provide insights into how JWT tokens are utilized within each framework and help you make a transition from one to another easier.

Understanding the Basics of JWT Tokens

At the core of JWT tokens lies a simple yet powerful structure that encompasses all the necessary information for secure authentication and authorization. Let’s dive into the basics of JWT tokens and explore their three essential components: the header, the body, and the signature.

1. Header

The header of a JWT token contains metadata about the token itself and the algorithms used to secure it. It typically consists of two parts: the token type, which is always “JWT,” and the signing algorithm employed, such as HMAC, RSA, or ECDSA. This header is Base64Url encoded and forms the first part of the JWT token.

2. Body (Payload):

The body, also known as the payload, carries the actual data within the JWT token. It contains the claims, which are statements about the user and additional metadata. Claims can include information like the user’s ID, name, email, or any other relevant data. The payload is also Base64Url encoded and forms the second part of the JWT token.

3. Signature

The signature is the crucial component that ensures the integrity and authenticity of the JWT token. It is created by combining the encoded header, the encoded payload, and a secret key known only to the server. The signature is used to verify that the token has not been tampered with during transmission or storage. It acts as a digital signature and prevents unauthorized modifications to the token. The signature is appended as the third part of the JWT token.

Self-Contained and Secure

One of the significant advantages of JWT tokens is their self-contained nature. Since all the necessary information is embedded within the token itself, there is no need for additional database queries or session lookups during authentication and authorization processes. This inherent characteristic contributes to improved performance and scalability.

To verify the authenticity and integrity of a JWT token, the recipient needs access to the public key or shared secret used to generate the signature. By retrieving the public key or shared secret, the recipient can verify the token’s signature and ensure that no tampering or unauthorized modifications have occurred. This mechanism provides a robust security layer, assuring that the token’s contents can be trusted.

User Roles in JWT Tokens

JWT tokens can also include user roles as part of their payload. User roles define the permissions and privileges associated with a particular user. By including this information in the JWT token, applications can determine the user’s authorization level and grant or restrict access to specific resources or functionalities accordingly. This granular approach to authorization allows for fine-grained control over user permissions within the application.

In the upcoming sections, we will explore how different frameworks incorporate these fundamental JWT token concepts into their authentication and authorization workflows. Understanding the core principles behind JWT tokens sets the stage for a comprehensive comparison, enabling us to evaluate the strengths and nuances of each framework’s implementation.

Example application

The same example application is made with different runtimes. It contains a couple of endpoints, they all require a valid token before they should be executed. One of the endpoints requires that the token contains the role of administrator.

GET /protected/user -> Hello username
GET /protected/admin -> Protected Resource; Administrator Only

The tokens utilised in our example are sourced from Keycloak, a reliable and widely adopted Authorization provider. Keycloak offers various standard flows for obtaining these tokens, catering to diverse authentication scenarios.

One of the commonly employed flows is the authorization code flow, which involves user interaction through dedicated screens provided by the Authorization provider. Users are prompted to log in and provide their credentials, following which Keycloak generates the necessary tokens for authentication and authorization purposes.

Alternatively, Keycloak supports a username and password-based approach where users can submit their credentials to a designated endpoint. This method allows Keycloak to validate the provided information and issue the relevant tokens required for subsequent authentication and authorization processes.

For our example, a custom realm with a configuration that is suitable for all our runtimes is created by setup_jwt_example.py and can be found in the directory https://github.com/rdebusscher/Project_FF/tree/main/jwt/keycloak. The script prepares the realm and a OpenId Connect client so that in response to a valid user name and password combination, a JWT token with the roles of the user is returned. It creates also two users, one of them having the admin role.

The Python script test_jwt_example.py can be used to test out the solution in each of the runtimes. It calls both endpoints with the two users that are defined. And so, one of the calls will result in an error since the non-administrator user is not allowed to call the administrator endpoint.

Runtimes

The different runtimes are discussed in part 2 and part 3 of this series.

Part 1: Introduction (this one)
Part 2: Payara, Spring Boot and Quarkus
Part 3: Ktor and Atbash Runtime
Part 4: Discussion and conclusion

Training and Support

Do you need a specific training session on Jakarta EE, Quarkus, Kotlin or MicroProfile? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
Best Practices Kotlin Ktor

Centralised Exception handling within Kotlin Ktor

A centralised Exception handling within a web framework is an important aspect of your application. The functionality that is provided by your code for the client is probably the most important aspect. But not every request can be successfully processed. Not to mention that your application can receive malicious requests that should be handled gracefully.

A centralised handling is highly recommended so that you can handle these exceptional cases uniformly. In this blog, I’ll explain how you can achieve such a centralised handling within Kotlin Ktor that is outside of your business code.

Types of Errors

There are several types of Errors that your applications should handle. Let us go over each of them and how you can handle them. In the next step, we will then handle all Exceptions in a central place.

Missing URL parameters

Within Ktor, we define the template URLs that we like to handle. many of these URLs have a placeholder in them so that we can handle similar requests by the same piece of code. But the values for these placeholders might be missing or wrong.

Let us consider the URL that we define to retrieve detailed information about a product of our webshop based on its id.

GET /product/{productId?}

And we can have multiple URLs in our application that all have a productId parameter in the template. We can use a helper method to retrieve this parameter from the URL and call it in each case we have thus productId parameter.

    fun extractProductId(context: PipelineContext<Unit, ApplicationCall>): String {
        return context.call.parameters["productId"] ?: throw HasMissingParameterException("Missing 'productId' in URL")
    }

The HasMissingParameterException is a RuntimeException that we will handle later on. And we can create such a helper function for each template variable our application has and group them in a Kotlin file.

The reason we have defined the parameter as optional (the ? at the end of the name) allows us to handle the absence of the parameter. The client may, on purpose or by accident due to a problem in their own code, send requests like /product/. Due to the optional marker, the same code will be executed within our Ktor application and a HasMissingParameterException will be thrown.

In the next section, we will see how these HasMissingParameterException exceptions will be handled.

Incorrect ids

Related to the previous paragraph where we handled the productId, we should verify if the productId is a valid value.

If we always expect that the productId is a numeric value, a positive long value for example, we can incorporate this check into the extractProductId function we used earlier. Instead of returning the value immediately as in the above example, we can try to convert the String value to a Long.

If that fails, we can throw another exception to indicate this problem.

Further on in the processing of our request, we need to check if the productId that is specified is an existing value. It might be a positive value but the number might not be a product that exists in our database.

When our code finds this out, it can throw an exception to indicate the entity is not found. Our generic Exception handling can pick this up and responds with the HTTP status 404.

See the example application and the BookEntityNotFoundException class for an example of this scenario. (project on GitHub )

Business logic error

The last category of errors we need to handle is business logic errors. If the request that we are handling violates one of the business rules we have defined, an order cannot be made for a customer that has more than x euro open invoices, and the request should be aborted.

Our code can throw an exception at this point and our generic exception handling will do the rest. Our code doesn’t need to know how to handle it, it just needs to show a custom exception.

All these exceptions should extend from a common exception, like BusinessException so that we can handle these cases easily in our ExceptionHandler.

Sending errors to the client

Now that we have proper Exceptions thrown at the various points in our code, we need a central place where we can handle them. This way, our business logic is not tight to the handling of the requests and can be reused in other scenarios.

We can install an ExceptionHandler through the StatusPage functionality of Ktor. So make sure you have added this module to your application. But it is not much trouble to add it later on as it just required the Maven artefact.

    <dependency>
        <groupId>io.ktor</groupId>
        <artifactId>ktor-server-status-pages-jvm</artifactId>
        <version>${ktor_version}</version>
    </dependency>

Installing the handler within the Ktor framework is also very straightforward. You need the following snippet to forward the exception handling to the handle() method

install(StatusPages) {
    exception<Throwable> { call, cause ->
        ExceptionHandler.handle(call, cause, developmentMode)
    }
}

The development mode variable is a boolean that indicates if you started the application in development mode. It can be used to give more information in the response or in the log about the exception that occurred. When in production mode, the code makes sure that no internal information is returned to the caller that might give some hints to malicious users about your application internals and how it can be abused. In the end, it is up to you and the additional components you use in your environments how to deal with Exception.

But don’t print a stack trace for each business logic or conversion error from String to Long when retrieving parameters. As these exceptions are thrown to easily handle error responses and are not some kind of error in your code, but are expected.

What does the handle method look like?

   when (cause) {
        is EntityNotFoundException -> {
            // for the cases the user specified an URL parameter where the id doesn't exist.
            call.respond(
                HttpStatusCode.NotFound,
                ExceptionResponse(cause.message ?: cause.toString(), HttpStatusCode.NotFound.value)
            )
        }

        is BusinessException -> {
            // Some business logic error, status 412 Precondition Failed is appropriate here
            call.respond(
                HttpStatusCode.PreconditionFailed,
                ExceptionResponse(cause.message ?: cause.toString(), cause.messageCode.value)
            )
        }

        is ParameterException -> {
            // The client forgot to define a Path or Query parameter or used a wrong type (string and not a number)
            call.respond(
                HttpStatusCode.BadRequest,
                ExceptionResponse(cause.message ?: cause.toString(), HttpStatusCode.BadRequest.value)
            )
        }
        // We can have other categories
        else -> {
            // All the other Exceptions become status 500, with more info in development mode.
            if (developmentMode) {
                // Printout stacktrace on console
                cause.stackTrace.forEach { println(it) }
                call.respondText(text = "500: $cause", status = HttpStatusCode.InternalServerError)
            } else {
                // We are in production, so only minimal info.
                call.respondText(text = "Internal Error", status = HttpStatusCode.InternalServerError)
            }
        }

This is the code from the example project I prepared for this blog. It can be adapted to the needs of each specific application. The overall idea is that you have a specific HTTP status for each type of problem. The ParameterException, indicating a missing or wrong type for a path or query parameter results in status 400. EntityNotFoundException when an id is not found in the database and BusinessExceptions indicating a business rule violation results in status 404 and 412 respectively.
All other exceptions result in status 500 and when in development mode, more info is available in the response and in the log.

The body of those errors is a JSON object that is created from the data class ExceptionResponse. It contains a code and a short message so that the client has some information about what went wrong. Since most of the HTTP status codes returned by this handler are in the 400 range, meaning they are client errors. the client did send a request which was incorrect so it should receive some feedback on what was wrong.

Conclusion

Proper Exception handling is an important part of your application. As a best practice, not every method should deal with returning the proper response as that would mean that your business code knows about the type of client. Instead, you should throw Exceptions, all having a specific parent that indicates the type of problem that occurred. Types are related to the parameters of the request, non-existing ids that are provided, or violations of business rules. Other types of problems can exist depending on the application. Regardless of the number of types, a centralised exception handler, installed as StatusPage handler within Ktor, the correct HTTP status and response body can be sent back from a single method. And don’t forget to include some code and a short description of the problem so that the client knows what went wrong. Without giving away too many details of the internals of your application in case a malicious user tries to figure out your application by sending random or incorrect requests on purpose.

You can find an example of how you can implement this strategy in the project located at https://github.com/rdebusscher/kotlin-projects/tree/main/ktor-exceptions.

Training and Support

Do you need a specific training session on Kotlin, Jakarta EE or MicroProfile? Or do you need some help in getting up and running with your next Ktor project? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
JAX-RS Ktor

Annotation-based definition of routes in Ktor

As you might know or have seen in my previous blog, the declaration of the routes and HTTP methods is done programmatically within Kotlin Ktor.

If you are used to specifying this information through annotation in Jakarta EE, it might feel a bit awkward. Although, due to the ‘trailing lambdas’ construct, this almost looks and feels like an annotation.

But actually, you can have something similar to annotations from the Jakarta EE world, with a little custom code.

Standard Ktor way

As mentioned, the trailing lambdas solutions allow you to define the routes in a very similar way to Jakarta EE. For each HTTP method, there exists a method with the same name that accepts 2 parameters. The part of the URL and a lambda which is executed when there is a request that matches the URL defined in the first parameter.

In this snippet, there are a few examples

    get("/hello/{name?}") {
        val name = call.parameters["name"] ?: return@get call.respond(
            call.respond(
                HttpStatusCode.BadRequest,
                "Missing name parameter"
            )
        )
        val language = call.request.queryParameters["language"] ?: "en"
        call.respondText("Hello $name with language $language")
    }

    post("/person") {
        val person = call.receive<Person>()
        call.respondText("POST received with name ${person.name} and age ${person.age}")
    }

The required values from the request like path and query parameters or the body of the request can be retrieved from the call object passed into the lambda. This same object is also used to return the result to the client.

And this declaration, although programmatically and with no reflection or scanning required, is similar to the way we would define those methods in Jakarta EE.

@GET
@Path("/hello/{name}")
public String sayHello(@PathParam("name") String name, @QueryParam("language") String language) {

}

Annotations solution

As mentioned in the introduction, with a little help of custom code, you can have a solution that uses annotations on functions and is very similar to the Jakarta EE approach.

Let us first look at an example

@GET("/{name}")
suspend fun greeting(call: ApplicationCall, name: String, @DefaultValue("en") language: String?) {
    // function parameter names must match either variable name in URL pattern or is matched against query parameters
    call.respondText("Hello $name with language $language")
}
}

You can see that we define a method that has a GET annotation. The value of the annotation is a reference to a variable name that we also have as the function parameter name. But we can also retrieve query parameters by just defining them as parameters.

When we define a parameter as optional like in the example, the query parameter language is not required in the URL. If we do not specify it as optional and the user calls the endpoint without specifying the query parameter, you get an exception.

In case the parameter is optional, we can define a default value by using the custom annotation @DefaultValue as we see in the example.

Besides the possibility to catch path and query parameters as function parameters, we can also capture the body of the request. An example can be seen in the next snippet

@POST("/person")
suspend fun savePerson(call: ApplicationCall, person: Person) {
    // The body can be retrieved by defining a function parameter. Conversion from JSON happens automatically.
    call.respondText("POST received with name ${person.name} and age ${person.age}")

}

Here we capture the body of the Post request and convert the JSON to a Person instance.

In all cases, we also pass the ApplicationCall instance to the method so that we can perform more logic based on request values and send the response back to the client.

The last thing we need to do is indicate where the methods can be found that have those annotations. Since Ktor doesn’t make use of a scanning mechanism, we need to provide the classes with those annotated functions.

The custom code wraps these functions in the regular Route definition of Ktor. This statement using the custom function processRoutes processes the functions that have those custom annotations and makes it transparent for Ktor.

routing {
    processRoutes(HelloResource(), JsonResource())

}

The code is based on a gist from Thomas van den Bulk.

Conclusion

If you like to use the annotation-based configuration of endpoints in a similar way as Jakarta EE, you can use the code that is showcased in the project Ktor-annotated available within this GitHub repository.

A very small set of functions allow you to define annotations on Kotlin functions that define the Http method and call the function when a matching request comes in converting path and query parameters to the function parameters.

It can help those familiar with the way of defining REST endpoints within Jakarta EE but also simplifies handling path and query parameters for those familiar with Ktor.

Training and Support

Do you need a specific training session on Kotlin, Jakarta EE or MicroProfile? Or do you need some help in getting up and running with your next Ktor project? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

Categories
JAX-RS Kotlin Ktor

Why don’t you create your next Backend application with Kotlin and Ktor?

For a long time, Kotlin was on my list of things I wanted to learn more about. In the second half of 2022, I finally got the chance to learn the language and started using Ktor, the ‘web application runtime’ for Kotlin.

Kotlin

Kotlin is a modern programming language that offers several benefits over Java. Firstly, Kotlin supports object-oriented, functional, and procedural programming constructs, making it a versatile language for a wide range of projects.
Secondly, Kotlin is fully interoperable with Java, which means that developers can easily use Kotlin in conjunction with existing Java codebases. Additionally, Kotlin offers support for several concepts, such as data classes, structured concurrency, and virtual threads, which were not available in Java until much later. Kotlin also offers null safety features, which help prevent common errors that can arise due to null pointer exceptions.
Another significant benefit of Kotlin is its compact and highly readable grammar, which makes code more concise and easier to understand. Finally, Kotlin offers extension functions, which allow developers to add new functions to existing classes, making it easier to write reusable code.

Kotlin is a modern, powerful, and highly expressive programming language that helps developers write better code faster and more efficiently.

We will encounter various powerful constructs of Kotlin in this blog series called “project FF”. In this Framework Face-off, I will compare several Java application runtimes, including Spring Boot and Quarkus, with Jakarta EE and how you implement Enterprise functionality.

But today, I give already two examples of the power of Kotlin.

Kotlin extension functions

Kotlin extension functions allow you to add new functions to an existing class without having to inherit from that class or modify its source code.
In other words, extension functions let you extend the functionality of a class in a more flexible and modular way. This can make your code more readable and easier to maintain.

Extension functions are similar to static utility methods, but with the added benefit of being able to call them as if they were instance methods of the class, they are extending. This can make your code more concise and easier to understand.

fun String?.hasSpaces(): Boolean {
   val found = this?.find { it == ' '}
   return found != null
}

fun main() {
   val data = "A sentence with spaces in of course"
   println(data.hasSpaces()) // True

   val data2 = "word"
   println(data2.hasSpaces()) // False
}

For a detailed explanation of this code, I refer you to the Kotlin tutorials. In short, the hasSpaces function is defined on a nullable String type. The function body checks if we find a space in the string value. From that point on, we can use the hasSpaces on a String as you can see in the main function.

Creating an interceptor

You can create an interceptor in plain Kotlin using lambdas and high-order functions. Functions and lambdas are first-class citizens and are treated equally as data objects.

inline fun timeBlock(block: () -> T): Pair {
   val startTime = System.nanoTime()
   val result = block()               // execute lambda
   val endTime = System.nanoTime()
   return Pair(result, endTime - startTime)
}

fun main() {
   val number = 121
   val (isPrimeValue, time) = timeBlock {
      (2..number/2).none { number % it == 0 }
   }

   println("Result of prime check for $number is $isPrimeValue ")
   println("Execution took $time nanoSeconds ")
}

Here we define a function that accepts a lambda and we time the execution of the lambda. The function returns the result of the lambda and the time it took.

As example, we time the calculation to check if a value is a prime value. Also here you can see the power of Kotlin but still keep it readable.

(2..number/2).none { number % it == 0 }

defines that we go from value 2 until the half of the value we need to check and there should be no value where the modulo of the value with the number is 0.

Ktor

Ktor is a lightweight, asynchronous web server that easily allows you to define what is needed. In the blog “Time is code, My friend”, Ktor performed really well and performed as good as Quarkus which has a very elaborate and complex pre-processing system to achieve such a fast startup.

Since it is highly modular, you can keep the runtime, and memory usage very low. And since there are modules available for each major topic or framework, you can create applications that integrate with them very easily.

Also, Ktor is created with Kotlin Coroutines support built-in. Kotlin Coroutines provide you with solutions for asynchronous non-blocking programming, structured concurrency and solutions similar to Java Virtual Threads. Things that will only appear in Java in the coming versions but are already available for many years within Kotlin.

Ktor Example

In this blog, I’ll show you how you can create an application with endpoints that handle JSON payload.

You can generate a Maven or Gradle project using the Ktor Project Generator web page or directly from within IntelliJ IDEA.

By default, the Netty engine is selected to handle the HTTP request handling but you can select other engines if you like.

Under the plugin section, make sure you select the ‘kotlinx.serialization‘ plugin to have support for JSON. This automatically brings in the routing plugin which is responsible for calling a certain Kotlin function for a URL pattern and the Content Negotiation plugin that sets header values accordingly to the requirements like JSON types.

The generated project contains a main function within the Application.kt file which starts Netty and an extension function that define the configuration of the modules. This allows for a clear separation of each functionality and quickly lets you find each setting.

To have full JSON support, we don’t need to configure anything, just activate the plugin. The Maven plugin in this case makes sure that any class that is annotated with @Serializable can be used at runtime without the need for any reflection. This makes the solution native compile friendly. This a topic we will discuss later in more detail.

For the routing, we call a function that corresponds to the HTTP method we want to support. It has a URL pattern and lambda expression as a parameter. This way, we can define the function for each URL in a very concise and readable way.

    get("/") {
        call.respondText("Hello World!")
    }

The entire example can be found at the Project FF Github repository: https://github.com/rdebusscher/Project_FF/tree/main/ktor-started/JAX-RS

Conclusion

In a time where modular runtimes that are small and take up little memory are important for some people, Kotlin and Ktor are ideal solutions. Kotlin offers many benefits over Java and offers a compact syntax but still highly readable. It supports many important features already for years that are introduced only recently or planned in Java.

Ktor is the natural solution if you choose Kotlin and need to write a backend application. Although you can combine Kotlin with Spring Boot and Quarkus, Ktor is a lightweight solution that has the same functionality available through its modules and is faster at runtime.

Training and Support

Do you need a specific training session on Kotlin, Jakarta EE or MicroProfile? Or do you need some help in getting up and running with your next Ktor project? Have a look at the training support that I provide on the page https://www.atbash.be/training/ and contact me for more information.

This website uses cookies. By continuing to use this site, you accept our use of cookies.  Learn more