Load balancing Quarkus

Configuring the server
1

Hello,

I have read the latest documentation and when it comes about clustering and load balancing it only evoques the manipulation to do on a Wildfly distribution and not at all what to do on a Quarkus one.

So my question is pretty simple, what do I need to setup in Keycloak in order to load balance Keycloak properly?

Some data:

  • Using Keycloak 18.0
  • Load balancing used : haproxy (I manage that part).

Thanks

2

Keycloak uses the infinispan cache to share sessions between instances. Assuming you have that working properly, there is no special setup you need in your load balancer.

3

Meaning I can only use the configuration stated in the documentation regarding infinispan?

What about the external PostgreSQL database? Can several instances point to the same database? Should they be replicated ? If yes, how? Active or passive?

Thanks

4

If you haven’t already, I would recommend looking at the “Server” guides here: Guides - Keycloak
There are sections there regarding cache, database and reverse proxy. If one of those is unclear, post a specific question here.

5

That’s what I’ve read and all docs speaks about standalone.xml and domain.xml which doesn’t exists in Quarkus anymore and my question was about doing it with a Quarkus distribution.

6

Incorrect. The guides in the link I sent are specific to the Quarkus distribution.

If you have a configuration you’ve tried, please share it, and we can help you debug.

1 Like
7

I moved forward on this topic by continue reading and manipulating the same time.

What I have done so far is to use your gist giving a proper cache-ispn.xml file and make Keycloak use it properly.
Looking at the table jgroupsping I have one row containing my host.

Embedded infinispan seems to start properly as per Keycloak log:

2022-08-07 15:03:38,534 DEBUG [org.infinispan.quarkus.hibernate.cache.QuarkusInfinispanRegionFactory] (JPA Startup Thread: supplier-store) Starting Infinispan region factory
2022-08-07 15:03:38,560 DEBUG [org.hibernate.internal.SessionFactoryImpl] (JPA Startup Thread: supplier-store) Instantiating session factory with properties: {hibernate.hbm2ddl.auto=update, hibernate.allow_update_outside_transaction=false, hibernate.connection.handling_mode=DELAYED_ACQUISITION_AND_RELEASE_BEFORE_TRANSACTION_COMPLETION, hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect, javax.persistence.transactionType=JTA, hibernate.bytecode.allow_enhancement_as_proxy=true, hibernate.xml_mapping_enabled=false, hibernate.multiTenancy=NONE, hibernate.jdbc.wrap_result_sets=false, hibernate.ejb.persistenceUnitName=supplier-store, hibernate.temp.use_jdbc_metadata_defaults=true, hibernate.transaction.coordinator_class=class org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl, hibernate.connection.datasource=io.agroal.pool.DataSource@563a5746, hibernate.show_sql=false, hibernate.connection.provider_disables_autocommit=true, hibernate.cache.region.factory_class=org.infinispan.quarkus.hibernate.cache.QuarkusInfinispanRegionFactory, hibernate.order_updates=true}
2022-08-07 15:03:38,580 DEBUG [org.infinispan.quarkus.hibernate.cache.QuarkusInfinispanRegionFactory] (JPA Startup Thread: keycloak-default) Starting Infinispan region factory
2022-08-07 15:03:38,580 DEBUG [org.hibernate.internal.SessionFactoryImpl] (JPA Startup Thread: keycloak-default) Instantiating session factory with properties: {hibernate.allow_update_outside_transaction=false, hibernate.connection.handling_mode=DELAYED_ACQUISITION_AND_RELEASE_BEFORE_TRANSACTION_COMPLETION, hibernate.dialect=io.quarkus.hibernate.orm.runtime.dialect.QuarkusPostgreSQL10Dialect, javax.persistence.transactionType=JTA, hibernate.bytecode.allow_enhancement_as_proxy=true, hibernate.xml_mapping_enabled=false, hibernate.multiTenancy=NONE, hibernate.jdbc.wrap_result_sets=false, hibernate.ejb.persistenceUnitName=keycloak-default, hibernate.temp.use_jdbc_metadata_defaults=true, jboss.as.jpa.managed=false, hibernate.transaction.coordinator_class=class org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl, kc.query.deleteExpiredClientSessions=delete from PersistentClientSessionEntity sess where sess.userSessionId IN (select u.userSessionId from PersistentUserSessionEntity u where u.realmId = :realmId AND u.offline = :offline AND u.lastSessionRefresh < :lastSessionRefresh), hibernate.query.startup_check=false, hibernate.connection.provider_disables_autocommit=true, hibernate.connection.datasource=io.agroal.pool.DataSource@3d34d26b, hibernate.cache.region.factory_class=org.infinispan.quarkus.hibernate.cache.QuarkusInfinispanRegionFactory, hibernate.order_updates=true}

Instance is well detected aswell:

2022-08-07 15:03:39,754 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: core-keycloak-51068, Site name: null

However it stills argues that no remote store is configured for cache.

2022-08-07 15:03:39,854 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) No remote store configured for cache ‘sessions’
2022-08-07 15:03:39,854 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) No remote store configured for cache ‘clientSessions’
2022-08-07 15:03:39,855 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) No remote store configured for cache ‘offlineSessions’
2022-08-07 15:03:39,855 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) No remote store configured for cache ‘offlineClientSessions’
2022-08-07 15:03:39,856 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) Skipping pre-loading of userSessions from persistent storage
2022-08-07 15:03:39,859 DEBUG [org.keycloak.cluster.infinispan.CrossDCAwareCacheFactory] (main) No configured remoteStore available. Cross-DC scenario is not used
2022-08-07 15:03:39,869 DEBUG [org.keycloak.cluster.infinispan.InfinispanClusterProviderFactory] (main) Initialized cluster startup time to Sun Aug 07 15:03:38 CEST 2022
2022-08-07 15:03:39,871 DEBUG [org.keycloak.cluster.infinispan.InfinispanNotificationsManager] (main) Added listener for infinispan cache: work
2022-08-07 15:03:39,872 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserSessionProviderFactory] (main) Registered cluster listeners
2022-08-07 15:03:39,873 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanUserLoginFailureProviderFactory] (main) No remote store configured for cache ‘loginFailures’

Any idea why?

Per curiosity I also installed an external infinispan on the server, configured it using the same cache-container and jgroups section. Navigating to the console website I observe that my keycloak node appears, all cache files correctly created and not used with same message in Keycloak logs (no DC configured).

Any help would be more that welcome.