Clustering keycloak on 3 machines

Configuring the server
1

Dear all,

I have been bashing my head against this for a while now, but so far I have had no luck, and I am hoping someone could point me in the right direction.

  • I have three VPS’ each running docker (not Kubernetes)

  • I am running a keycloak container on each

  • I have a galera cluster shared between them for database

I am trying to configure with TCPPING, since I read that the default sync algorithm won’t work for the three machine configuration. But, I have tried the other syncs as well.

I am passing:


JGROUPS_DISCOVERY_EXTERNAL_IP="{{ docker_node_ip }}"
JGROUPS_DISCOVERY_PROTOCOL="TCPPING"
JGROUPS_DISCOVERY_PROPERTIES="initial_hosts={{ docker_node_1 }}[7600],{{ docker_node_2 }}[7600],{{ docker_node_3 }}[7600]"
JAVA_OPTS="-Dkeycloak.profile.feature.account_api=enabled -[Djboss.node.name](http://djboss.node.name/)={{ docker_node }} -[Djboss.tx.node.id](http://djboss.tx.node.id/)={{ docker_node }} -Djboss.bind.port=7600 -Djboss.bind_port=7600 -Djboss.bind_addr={{ docker_node_ip }} -Djboss.bind.address={{ docker_node_ip }} -Djboss.bind.address.private={{ docker_node_ip }}"

As configuration via ansible. But, when I run this each node exits with no error at:

Setting JGroups discovery to TCPPING with properties {initial_hosts=>xxx.xxx.xxx.184[7600],xxx.xxx.xxx.185[7600],xxx.xxx.xxx.186[7600]}

Does anyone have any idea what I’m doing wrong?

Many thanks,

Marcus

2

Are you using Keycloak WildFly or Quarkus?

Is there a specific requirement to use TCPPING instead of something like JDBCPING which is easy to understand and configure?

3

WildFly, I believe.

Using TCPPING because I tried JDBCPING and couldn’t get that to work either. I don’t mind which mechanism to use!

4

For clarity, I’ve tried JDBC_PING, and passed the connection details:

Setting JGroups discovery to JDBC_PING with properties {connection_url=>jdbc:mysql://xxx.xxx.xxx.188:3306/keycloak;connection_username=>keycloak_user;connection_password=>redacted;connection_driver=>com.mysql.jdbc.Driver;}

Also tried with com.mysql.cj.jdbc.Driver

In all cases JDBC_PING complains about these very settings being missing…

15:06:15,070 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 60) MSC000001: Failed to start service org.wildfly.clustering.jgroups.channel.ee: org.jboss.msc.service.StartException in service org.wildfly.clustering.jgroups.channel.ee: java.lang.IllegalStateException: java.lang.IllegalArgumentException: Either the 4 configuration properties starting with 'connection_' or the datasource_jndi_name must be set
	at org.wildfly.clustering.service@20.0.1.Final//org.wildfly.clustering.service.FunctionalService.start(FunctionalService.java:66)
	at org.wildfly.clustering.service@20.0.1.Final//org.wildfly.clustering.service.AsyncServiceConfigurator$AsyncService.lambda$start$0(AsyncServiceConfigurator.java:117)
	at org.jboss.threads@2.3.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
	at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at org.jboss.threads@2.3.3.Final//org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.IllegalStateException: java.lang.IllegalArgumentException: Either the 4 configuration properties starting with 'connection_' or the datasource_jndi_name must be set
	at org.jboss.as.clustering.jgroups@20.0.1.Final//org.jboss.as.clustering.jgroups.subsystem.ChannelServiceConfigurator.get(ChannelServiceConfigurator.java:116)
	at org.jboss.as.clustering.jgroups@20.0.1.Final//org.jboss.as.clustering.jgroups.subsystem.ChannelServiceConfigurator.get(ChannelServiceConfigurator.java:58)
	at org.wildfly.clustering.service@20.0.1.Final//org.wildfly.clustering.service.FunctionalService.start(FunctionalService.java:63)
	... 7 more
Caused by: java.lang.IllegalArgumentException: Either the 4 configuration properties starting with 'connection_' or the datasource_jndi_name must be set
	at org.jgroups@4.2.4.Final//org.jgroups.protocols.JDBC_PING.verifyConfigurationParameters(JDBC_PING.java:420)
	at org.jgroups@4.2.4.Final//org.jgroups.protocols.JDBC_PING.init(JDBC_PING.java:102)
	at org.jgroups@4.2.4.Final//org.jgroups.stack.ProtocolStack.initProtocolStack(ProtocolStack.java:854)
	at org.jgroups@4.2.4.Final//org.jgroups.stack.ProtocolStack.init(ProtocolStack.java:842)
	at org.jgroups@4.2.4.Final//org.jgroups.JChannel.<init>(JChannel.java:164)
	at org.jboss.as.clustering.jgroups@20.0.1.Final//org.jboss.as.clustering.jgroups.JChannelFactory.createChannel(JChannelFactory.java:116)
	at org.jboss.as.clustering.jgroups@20.0.1.Final//org.jboss.as.clustering.jgroups.subsystem.ChannelServiceConfigurator.get(ChannelServiceConfigurator.java:96)
	... 9 more

Obviously I’m missing something, but can’t see what…

5

Hi @mapkyca by just seeing the exception above, I couldn’t figure out the issue.

I’d suggest you to have a look at the keycloak-clustered that I’ve been working on. It uses JDBC_PING discovery protocol.

I wrote about it in this Medium article.

I hope, by checking how is the setup and configuration that I did, you can find what it’s missing in yours.

Best regards

1 Like