I deployed Keycloak on a Kubernetes cluster.
The service points to the DB as below:
apiVersion: v1
kind: Service
metadata:
name: external-mysql-db
namespace: iam-keycloak
spec:
clusterIP: None
ports:
- name: mysql
protocol: TCP
port: 3306
targetPort: 3306
selector: {}
---
apiVersion: v1
kind: Endpoints
metadata:
name: external-mysql-db
namespace: iam-keycloak
subsets:
- addresses:
- ip: "172.19.11.60"
ports:
- port: 3306
name: mysql
When running Keycloak with replicas = 1 , everything works smoothly.replicas = 2 or more , I can not login to the Administrator console.
Here is the Keycloak deployment:
keycloak-deploy.yaml
apiVersion: v1
kind: Service
metadata:
name: keycloak-svc
namespace: iam-keycloak
labels:
app: keycloak
spec:
type: ClusterIP
clusterIP: None
ports:
- name: http
port: 8080
targetPort: 8080
selector:
app: keycloak
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: keycloak
namespace: iam-keycloak
labels:
app: keycloak
spec:
replicas: 1
selector:
matchLabels:
app: keycloak
template:
metadata:
labels:
app: keycloak
spec:
containers:
- name: keycloak
image: quay.io/keycloak/keycloak:12.0.4
env:
- name: KEYCLOAK_USER
valueFrom:
secretKeyRef:
name: keycloak-secret
key: keycloak_user
- name: KEYCLOAK_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-secret
key: keycloak_pass
- name: PROXY_ADDRESS_FORWARDING
value: "true"
- name: DB_VENDOR
value: mysql
- name: DB_ADDR
value: external-mysql-db
- name: DB_DATABASE
value: keycloak_db
- name: DB_USER
value: keycloak_user
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-secret
key: db_password
- name: JGROUPS_DISCOVERY_PROTOCOL
value: dns.DNS_PING
- name: JGROUPS_DISCOVERY_PROPERTIES
value: dns_query=keycloak-svc
- name: CACHE_OWNERS_COUNT
value: "2"
- name: CACHE_OWNERS_AUTH_SESSIONS_COUNT
value: "2"
ports:
- name: http
containerPort: 8080
- name: https
containerPort: 8443
readinessProbe:
httpGet:
path: /auth/realms/master
port: 8080
initialDelaySeconds: 10
Anyone has a solution for this issues, please.
Simply multiplying the count of Keycloak instances is not enough, you have to configure a cluster - a Keycloak/Wildfly cluster, not related to K8s:
1 Like
Thank you for your help.
- name: JGROUPS_DISCOVERY_PROTOCOL
value: dns.DNS_PING
- name: JGROUPS_DISCOVERY_PROPERTIES
value: dns_query=keycloak-svc
- name: CACHE_OWNERS_COUNT
value: "2"
- name: CACHE_OWNERS_AUTH_SESSIONS_COUNT
value: "2"
And I have to add this annotation to Ingress:nginx.ingress.kubernetes.io/affinity: "cookie"
So I can run multi-replicas pods of Keycloak.
@truongnh was this enough?
thanks
@pierreozoux @dasniko Finally, I found the solution.
I have to insert the following script to the Keycloak Image: (keycloak.cli)
embed-server --server-config=standalone-ha.xml --std-out=echo
batch
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:write-attribute(name=owners, value=${env.CACHE_OWNERS_COUNT:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS_COUNT:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:write-attribute(name=owners, value=${env.CACHE_OWNERS_COUNT:1})
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:write-attribute(name=owners, value=${env.CACHE_OWNERS_COUNT:1})
echo Configuring node identifier
/subsystem=transactions:write-attribute(name=node-identifier, value=${jboss.node.name})
run-batch
stop-embedded-server
The Dockerfile:
FROM quay.io/keycloak/keycloak:12.0.4
RUN mkdir /opt/jboss/startup-scripts/
COPY keycloak.cli /opt/jboss/startup-scripts/
The environment for Keycloak container:
name: CACHE_OWNERS_AUTH_SESSIONS
value: "5"
name: JGROUPS_DISCOVERY_PROTOCOL
value: dns.DNS_PING
name: JGROUPS_DISCOVERY_PROPERTIES
value: 'dns_query=${keycloak-svc}.${namespace}.svc.cluster.local'
name: CACHE_OWNERS_COUNT
value: "5"
name: CACHE_OWNERS_AUTH_SESSIONS_COUNT
value: "5"
For more detail:
2 Likes
Wow, super! Thanks a lot for sharing all these details
Ok, it is maybe beyond the scope of this thread, but now I’m curious, why didn’t you use the officail operator?
Have a nice week!
hafidz
June 27, 2021, 5:32pm
7
Hi @truongnh I am wondering how to add the following script to keycloak image(cli) could you please tell me how to do that? really appreciate it, thanks!
Hi @hafidz
I used Dockerfile to build the Keycloak Image.
For detailed:
Let create a folder then push the Dockerfile and keycloak.cli in it:
The content of Dockerfile:
FROM quay.io/keycloak/keycloak:12.0.4
RUN mkdir /opt/jboss/startup-scripts/
COPY keycloak.cli /opt/jboss/startup-scripts/
Change directory to the folder contains Dockerfile and keycloak.cli. In order to build a new Image, run the following command:
docker build -t newkeycloak .
Done:)
Hi @truongnh
Is this setup did not require kubernetes service to connecting jgroups/infinispan between each pods ?
Sorry for my late reply.
@truongnh and all I am following your implemented solution for HA solution on kubernetes. I am using keycloak:14.0.0 and i have “CMD [”-b", “0.0.0.0”, “–server-config”, “standalone-ha.xml”]" in my docket file to start with ha mode. My question is this necessary to include keycloak.cli in my docket file ?
Second,In your keycloak.cli your defined CACHE_OWNERS_COUNT:1 . Will it be override from pod environment?
