Sync keycloak users with LDAP and external database using custom provider

We have setup Keycloak for our product solution in order to use it as the identity provider. We are going to import users from LDAP into the Keycloak database (Keycloak MySql production database). We also need to sync all the users from the Keycloak database (both local and LDAP) into an external database (MySql again) that is the database our product uses. We have a table in there to reflect the product users (systemusers).

Could you please help us understand what is the best way for syncing those users from Keycloak to our product database. I have read that there exist SCIM plugins that can do this, but we wont rely on that, we will implement something on our own. Therefore should we:

  • extend the User Storage SPI (for writing to our external product database whenever a Keycloak user -LDAP or local- is updated) or
  • extend the UserProfileProvider (the only documentation I found for this solution is in the javadocs)
  • extend a custom Event Listener SPI (for writing to our external product database whenever a Keycloak user -LDAP or local - is updated)

In the first case, we will implement the UserStorageProvider, UserLookupProvider and override the GetUserByUsername method so that if the current’s session user in keycloak does not exists in the external database, we create it.

In the last case, we will implement the EventListenerProvider and override the OnEvent method to trigger sync CRUD operations to our product database whenever events such as UPDATE_PROFILE happen.

The question I have is whether events will be triggered when Keycloak imports a user from LDAP too, or only when the user is updated through the admin console? Could you please tell me if there is way to detect these LDAP updates? With local users updated through admin, events such as UPDATE_PROFILE are triggered but with LDAP, I am not sure if events are created that will help us to trigger the CRUD operations in our product database.

Or if the EventListenerProvider extension is not the correct way forward, then is the User Storage SPI Provider the correct way to implement this synchronisation between the 3 data stores?

[Update]
We have chosen to go forward with the UserStorageProvider and and UserLookupProvider extensions and the UserStorageProviderFactory. This implementation looks up the user in keycloak and if the user is found calls our backend restful APIs to create the user in our external database. However the issue we currently phased is that the UserStorageProviderFactory create method is not being invoked and the custom provider is not triggered to do the CRUD operations. Any ideas why this is happening?
Or maybe if this is not the correct approach, could you please suggest any alternatives?

Any help will be greatly appreciated! Many thanks!

You have a 3-way synchronization? LDAP / Keycloak internal table / external MySQL?

Hi Carl!

Thank you for your help, yes actually Keycloak synchronises the users from LDAP and then we must create them from Keycloak to the external MySQL. This means that also any users created manually through the admin console UI, or synched from LDAP must be written to the external database.

I have also just came across the UserRegistrationProvider, could this be the solution to the sync issue between LDAP, keycloak and external database?

Many thanks,

Hi Carl!

Could you please let me know if you have any update or any recommendation on how to sync the imported LDAP users in the internal database to our external database? Any feedback would be greatly appreciated.

Many thanks,

Hi,

For the external database, I would put the burden of synchronization and cleanup on the app. When you come in with a valid token for the first time, create an account. Reap the account later based on cleanup rules (90 days no login, poll Keycloak daily for deletions.

For LDAP, I’d do one of two things. The first, I’d consider is to use the off-the-shelf LDAP user storage provider as the primary source of identity. That puts the burden of syncing Keycloak and LDAP on Keycloak. The 3-way sync problem becomes 2-way.

Another option is to make the LDAP source an external Identity Provider: an LDAP realm. This would use the previously-mentioned user storage provider. The difference would be that this is a read-only setup that would link to a second realm. This second realm would be the basis of the app integration.

Good luck,
Carl

1 Like