Keycloak running on docker cannot synchronize member cluster

1

Hi all,

I’m running Keycloak (15.1.1) in docker mode on two different hosts on AWS.
I’m using TCPPING protocol.
Followed : Keycloak Cluster Setup - Keycloak

PS: I can Telnet from one member to another member on port 7600 and get a response, so it’s not an issue with port binding, AWS Security group, etc

Unfortunately member cannot discover each other.
I get the following message in the logs :
10:54:48,376 INFO [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool – 58) 2009bd815bcb: no members discovered after 3003 ms: creating cluster as coordinator
10:54:49,520 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 59) ISPN000094: Received new cluster view for channel ejb: [2009bd815bcb|0] (1) [2009bd815bcb]
10:54:49,520 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 61) ISPN000094: Received new cluster view for channel ejb: [2009bd815bcb|0] (1) [2009bd815bcb]
10:54:49,513 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 60) ISPN000078: Starting JGroups channel ejb
10:54:49,523 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 60) ISPN000094: Received new cluster view for channel ejb: [2009bd815bcb|0] (1) [2009bd815bcb]
10:54:49,522 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 58) ISPN000094: Received new cluster view for channel ejb: [2009bd815bcb|0] (1) [2009bd815bcb]
10:54:49,530 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 62) ISPN000078: Starting JGroups channel ejb
10:54:49,531 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 62) ISPN000094: Received new cluster view for channel ejb: [2009bd815bcb|0] (1) [2009bd815bcb]
10:54:49,543 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 62) ISPN000079: Channel ejb local address is 2009bd815bcb, physical addresses are [0.0.0.0:7600]
10:54:49,552 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 60) ISPN000079: Channel ejb local address is 2009bd815bcb, physical addresses are [0.0.0.0:7600]
10:54:49,572 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 61) ISPN000079: Channel ejb local address is 2009bd815bcb, physical addresses are [0.0.0.0:7600]
10:54:49,584 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 58) ISPN000079: Channel ejb local address is 2009bd815bcb, physical addresses are [0.0.0.0:7600]
10:54:49,596 INFO [org.infinispan.CLUSTER] (ServerService Thread Pool – 59) ISPN000079: Channel ejb local address is 2009bd815bcb, physical addresses are [0.0.0.0:7600]

Configuration

100.64.6.5[7600],100.64.134.5[7600]

Any idea ??

Regards,

HA

2

Not sure, but it seems that tcpping is not working as it should, so I’d double check the configuration.

Which environment variables are you using? You could also double check if the cli/TCPPING.cli is correctly being running, I suppose this appears in the log.

3

Hi,

First, thanks for your help !!

In the docker-compose file, I put
#IP address of this host, please make sure this IP can be accessed by the other Keycloak instances
JGROUPS_DISCOVERY_PROTOCOL: TCPPING
JGROUPS_DISCOVERY_EXTERNAL_IP: ${PRIMARY_IP}
JGROUPS_DISCOVERY_PROPERTIES: initial_hosts=“${PRIMARY_IP}[7600],${SECONDARY_IP}[7600]”
KEYCLOAK_CACHE_OWNERS_COUNT: 2
KEYCLOAK_AUTH_CACHE_OWNERS_COUNT: 2

I think the issue is related to the fact Keycloak is running on docker, and the public interface is not “exposed” correctly…
Maybe I wrong…

Regards,

HA

4

Under docker-compose, you’d probably want another discovery method, not TCPPING. Probably DNS_PING is better suited.

JGROUPS_DISCOVERY_PROTOCOL=dns.DNS_PING
JGROUPS_DISCOVERY_PROPERTIES=dns_query=<docker_compose_service_name>

docker_compose_service_name is the name your service is run under on docker-compose.

Suppose your docker-compose looks like that:

version: '3.8'
services:
  keycloak:
    image: keycloak_image
    environment:
    - JGROUPS_DISCOVERY_PROTOCOL=dns.DNS_PING
    - JGROUPS_DISCOVERY_PROPERTIES=dns_query=keycloak
    deploy:
      replicas: 3

I’m not sure that works, as I usually run keycloak on kubernetes, but I suppose you could try.

5

I am not 100% sure if TCPPING could work on aws fargate. Can you try JDBC_PING ?