Using Keycloak and Spring OAuth2 to manage users and 3rd party integrations — An Evaluation

Summary

This article discusses how to create and manage users and allow them to integrate a variety of 3rd-party accounts within one application using an external IAM service responsible for general user management and Spring Security OAuth2 for establishing external connections.

Today follows up on our previous article, where we already configured Spring Security OAuth2 to establish 3rd-party connections on behalf of a user.

The following evaluation focusses on the challenges of keeping this setup minimal, expandable and maintainable as complexity increases while making decisions on how we want to proceed in the future. We are still in the process of finding a solution we are truly satisfied with, we will get there incrementally.

It will also contain a lot of problems encountered, realising the preferred solution-sketches came with unexpected limitations as research deepened. It will also focus on that aspect (e.g. this is not a Keycloak tutorial, there are plenty of great resources on that available already).

Disclaimer: Feedback is much appreciated. I am by no means an expert presenting proven best-practices one can adapt without second guessing. I am just trying my best to tackle these challenges, learn and improve along the way and thought it may be interesting to share the thought process behind the decision making regarding the project.

1 The use case

As our project grew, we introduced new features that are standalone and do not require 3rd-party connections. Our application is no longer (just) an Interface connecting external services. Some users might be only interested in using said standalone features, thus we now need actual user-accounts.

Also we want to create our own data to persist relations between data from external systems.

For this, our existing setup would have been sufficient, simply create users referencing the external user some of the data came from. We did that for some time but decided to move away from it as the requirement mentioned above got specified.

Also, some changes to certain implementations require us to find a new way to refresh expired 3rd-party access tokens independent of the Spring Security layer.

In short, our requirements to user behaviour look as follows:

• A user should have access to a fully functional standard registration/login experience
• A user should be able to establish 3rd-party connections associated with their user account once and remove it on demand.
• A user should not be required to re-log-into the 3rd-party connections often (only theoretically if their Refresh Tokens for their API are expired)

2 Discarded solution

2.1 SAML?

I briefly looked into the SAML 2.0 protocol during research. Besides the modern security standard shifting from SAML to OIDC/OAuth2.0 regarding SSO, it seemed as if letting the SPs communicating with each other was not part of the deal.

While it can be used to configure secure SSO for a user to access their Ms/GitHub/etc. account through one Login provided by e.g. Keycloak as the IP, the aspect of our SP (A) needing to securely access data from SP (B) seemed to not be covered. Correct me if I overlooked something/ did not dig deep enough during this research.

Additionally, SAML features are often hidden behind a paywall/special enterprise licensing models in the external systems. We would not want to make having paid for another service be a requirement for using ours.

Alright, let’s move on to the next one!

2.2 Let Keycloak handle the user and all their connections!

Keycloak is lightweight, easy to configure and provides many features we would not want to implement ourself out of the box. It can be configured to be used for authorising requests to our application within minutes.

In short (basic testing setup), simply create a realm for your application, allow self-registration, create a client within that realm and register the data for that client in your Spring Security OAuth2 protected service:

fun keycloakClientRegistration(): ClientRegistration { 
   val baseUrl = keycloakConfig.baseUrl 
   val realm = keycloakConfig.realm 
   return ClientRegistrations 
     .fromOidcIssuerLocation("$baseUrl/$realm") 
     .registrationId(keycloakConfig.clientRegistrationId) 
     .clientId(keycloakConfig.clientId) 
     .clientSecret(keycloakConfig.clientSecret) 
     .scope("openid", "offline_access", "profile") 
     .clientName(keycloakConfig.label) 
     .build() 
}

Or alternatively use the Keycloak plugin for Spring.

And bam — you integrated Keycloak. On Authentication success, simply create a user entity referencing the users Keycloak-Id and you are ready to link your own internal data to that user.

No need to cheese additional Authentication-protocols like basic-auth into your Spring-Secuity config to provide maximum flexibility to your Users. Just a little Css-styling and you have powerful and ready-to-go user-management up and running.

That was nice! Now could we even move all the other OAuth2-Logins to Keycloak, delete all the changes we made to be able to collect all our credentials and have a nice and simple standard Spring Security OAuth2 config which we could even define in our application.yaml?

What we could do:

• Link OAuth2-Accounts to our Keycloak accounts — not via the UI* but as a custom Identity Provider.

*only 3rd-part systems supporting OIDC on top of OAuth2 can be configured like this, plain OAuth2 logins might fail when trying to configure them via the UI and as an OIDC-Provider in Keycloak — at least that happened when I briefly tested it with a non-OIDC provider.

• This also means that you have to write a little code for Keycloak for every non-OIDC Provider you want to integrate, completely doable but keep that in mind. Instructions on how to achieve that can be found here
• Once you linked a third-party account (via the account linking API, which provides a user experience comparable to what the User sees during regular OAuth2-Logins) you can fetch an Access Token for the API of that service as described here

Problems:

While researching I found some threads describing problems with Keycloak only holding an initial AccessToken retrieved on user login. As these are short lived, they expire after a short amount of time, requiring a re-link of the system. The Keycloak documentation states that this is the way to go when wanting to refresh expired AccessTokens (link)

Re-Linking the accounts starts the same redirect-dance-flow as initially connecting an account.

What we would rather like to use is an endpoint that provides us with an always-valid AccessToken. Refreshing these should be handled by the service passing the token to us. That way we can avoid having to include the same “try-re-login-try-again” logic in every service that needs 3rd-party API access. After all, we are also looking for a way to improve developer experience.

This is the developer experience I want

However, I have not tested the re-link approach and how it behaves while attempting to do it silently without user interference in the backend. There were also other arguments for our use case that made us refrain from further investing in this approach.

You may have noticed the big red X in our sketch.

One important aspect is that we have to integrate systems that do not support OAuth2 for accessing their APIs and also On-Premise systems where we have no way of configuring OAuth2-Clients to use. Therefore we implemented connecting these via registering a PAT (Personal Access Token). That functionality is completely absent in Keycloak and would have to be fully implemented by us, including endpoints, data structures, UI and business logic.

Only using Keycloak to handle OAuth2 connections and handle the PATs somewhere else would go against our aim to encapsulate all of the connection logic in one dedicated service.

We are also not JavaEE developers and would rather work with our preferred stack if a significant amount of custom implementations is required to achieve our desired functionality.

Another drawback is that we would have to host keycloak and the database it uses, adding additional infrastructure effort.

Conclusion:

In defence of Keycloak: It’s a system that does what it is designed to do very well. For the primary use-case of using (external) Identity Providers to create, authenticate and authorise users, their implementation choices do make a lot of sense. Also as a general remark, you don’t need to keep track of and/or store RefreshTokens for that use case.

Keycloak might also be a little bit overpowered for the current stage of our product. We want to maintain flexibility for switching IAM-systems in the future. For now, a cloud-based even more lightweight IAM system would be perfect, in the future a self-maintained Keycloak instance might be worth additional infrastructure efforts etc.

Links gathered/ entrypoints for further investigation:

• https://keycloak.discourse.group/t/is-it-possible-to-use-an-keycloak-accesstoken-to-get-access-to-the-microsoft-graph/6831
• https://keycloak.discourse.group/t/using-token-exchange-to-refresh-3rd-party-idp-access-token/17458

3 The plan

To wrap the last section up, for now we have too much uncertainty regarding committing to an external IAM system. Also some of our special requirements are not supported by free-to-use IAM solutions out of the box.

Hence we are keeping our Spring Boot Multi-Auth project as a wrapper/facade service for the whole subject of users and identity federation.

Let me introduce to you:

It can be implemented as a server functioning as an API gateway in a microservice infrastructure or an embedded auth-module in a modular/monolithic architecture.

In there we will keep all auth relevant customisations* to Spring Security, the integration of an external IAM system and custom implementations for PAT-registrations encapsulated.

* Note that we already extended Spring Security OAuth2 functionality to match parts of our requirements in the last article. The default SSOAuth2-Login functionality behaves pretty much the same as Keycloak’s implementation.

Under the hood, our module delegates auth-request of all different kinds to the right external providers or services. This way we can plug in external IAM systems without having to press our custom requirements into them, achieving a loosely coupled relation that can be replaced in the future. We will only have to worry about configuring new IAM systems regarding their domain and migrating users post-production-release.

The requirement for our IAM service is that is needs to implement the OIDC specification “Identity Provider”. We have to be able to register our authorization wrapper service as a client.

In the auth wrapper, we can simply create a client-registration for our identity provider and switch that configuration if required.

The setup also provides us with full control on how we want to handle Access and Refresh Tokens. With this approach, we can create our desired endpoint encapsulating refreshing connections so the rest of our services won’t have to worry about that.

Basically, we can implement our own token endpoint inspired by the Keycloak token-endpoint but with the additional feature long-lived connections.

The difference is that Spring also tracks RefreshTokens in addition to AccessTokens and can be configured to persist these tokens. This is what we need to establish long-lived connections that persist even if the user is not currently active/ his session gets interrupted for whatever reason (server update, expiration,…)

4 Conclusion

Especially in early product stages, flexibility is important.

Re-inventing the wheel from scratch because an external solution does not provide all of your requirements (or is pretty expensive license-wise) is probably as bad if not even worse than committing to an external (at that time eventually overpowered) tool too early in the process.

When wanting to reduce developer cost and implementation effort as well as keeping the system open to changes and enable temporary solutions (e.g. because you don’t (yet) want to pay for a license for “the perfect IAM system” for your use-case),

Using free/low-cost external Systems as a black box and quick way to enable standard functionality and handle customization in a wrapper sounds like a good way to go.

The wrapper approach using Spring Security OAuth2 and our own customizations helps us to reduce hard dependencies to external products and allows us to be flexible in the future, since we are implementing/configuring against an open standard (OIDC) rather than a concrete service/implementation for IAM.