I am trying to have two separate VMs with an instance of keycloak on each one. I need the two instances to sync user data with each other, and if one VM goes down, the other VM will pick up the slack and continue to perform until the first VM comes back online (and when it does come back online I need the user data to sync back to it). What is the best configuration to do this with, and any tips on how to configure it properly?
Keycloak does this by default using the Infinispan distributed cache. You’ll want to read the docs for “Standalone Clustered Mode” to set this up. Keycloak has lots of great defaults for setting this up easily.
https://www.keycloak.org/docs/latest/server_installation/#_standalone-ha-mode
Thanks for the response! I have mostly set up the standalone clustered mode with a shared DB, but not the load balancer yet. Once the load balancer is set up, would that be sufficient to see syncing and fault tolerance?
A load balancer with sticky-sessions is desirable, but not required. Standalone clustered mode with a shared database is enough.
So say I have 2 VMs with IP addresses 192.168.189.140 and 192.168.189.144, both running standalone clustered mode. How can I run and verify that the two instances are in fact fault tolerant?
You can crank up logging to DEBUG, and you’ll see the connections between the two instances in the logs. You can also enable the WildFly admin console, and you’ll be able to see it there.
Hmm I am not sure if I see any connections. I created a new VM to try again, would you happen to know where in the standalone-ha.xml file I should define a connection between the two VMs?
It is not something you define directly. Infinispan uses JGroups to discover the other nodes on the network.
Ah I see. So when I run keycloak on the first VM (192.168.189.144) Do I basically run:
sudo ./standalone.sh --server-config=standalone-ha.xml -Djboss.bind.address=192.168.189.144
and then the similar command for the second VM? I go to my browser, and do I simply type in http://192.168.189.144:8180 to access the keycloak console (offset is 100)? Or is there something else that must be configured?
I don’t have Keycloak running in a k8 cluster but I’m also using the Standalone Clustered Mode. When I restart the nodes I see logs similar to this:
14:00:32,589 INFO [org.infinispan.CLUSTER] (thread-14,ejb,keycloak-1) ISPN000094: Received new cluster view for channel ejb: [keycloak-2|25] (2) [keycloak-2, keycloak-1]
Maybe that helps you intentify if Infinispan discovered your nodes correctly.
Hmm I don’t see anything like that in my logs. I wonder if something else is the issue then
Did you set up jgropus for communication? If not, your instances will not be able to see each other.
Not for the standalone clustered, I thought that was only necessary if using sticky sessions?
nope, if you are doing cluster you will need to set up jgroups, maybe the easiest way is if you do JDBC ping, I presume that you have dedicated DB?
If you need example for clustering with a reverse-proxy / load-balancer fronted, just take a look at these examples: keycloak-project-example/deployments/local/cluster at main · thomasdarimont/keycloak-project-exam
Instead of JDBC_PING I’d recommend to use kube_ping / dns_ping if you run with Kubernetes
dns_ping if you control DNS or TCP_PING if you want to start with a fixed initial list of IPs (that can be expanded at runtime).
You can find some examples for setting this up here: GitHub - ivangfr/keycloak-clustered: keycloak-clustered extends Keycloak docker image. It allows t
Btw. the Keycloak Book and the admin guide recommends to use sticky sessions to reduce latency by leveraging cache locality.
Cheers,
Thomas
2 Likes
Yes, I have 2 VMs with a keycloak instance on each, and a 3rd VM which has postgresql downloaded and setup on it.
This is what my logs look like on devbox:
So it looks like now the VMs recognize eachother
I am wondering if that is fine, and if it is, am I missing some part of the database setup specific to keycloak
As thomasdarimont said you can use different ways of discovering the service. If the current setup is working for you then great. I just propose one of the solutions. Not sure what kind of cache do you use for users, but it will depend will the cache be synced inside the cluster. I’m guessing that you are searching for replicated cache?