Hi,
I have been trying to figure out how to persist the Infinispan cache as I want to preserve the user sessions over Keycloak restart. This topic has been discussed in several other threads but I am still struggling to get it working.
Threads I have been reading/following
Preserve login session over KeyCloak restart with Infinispan local file-based cache - Configuring the server - Keycloak
jGroups/Infinispan Bedevilling: No DB Table Created (Unable to persist Infinispan internal caches as no global state enabled) - Miscellanaeous - Keycloak
and more (I am a new user and can only put 2 links in a post)
My setup
1 Keycloak instance running on a docker container
Uses a PostgreSQL db running on another docker container
Keycloak version 11.0.3 (Its old… I know)
My goal
Using the built-in infinispan that comes with Keycloak with local file-based caching or DB caching without an external Inifinispan cluster
1 Like
What I have tried
Attempt 1 - Using Keycloak 11.0.3 and configure the built-in Infinispan cache to use file-based persistence (Ideas taken from - Preserve login session over KeyCloak restart with Infinispan local file-based cache - Configuring the server - Keycloak )
Result - Keycloak container starts without any errors but failed to persist user sessions after restarting the container. (The web front-end redirected to the login page after refreshing the page)
Created a Dockerfile with the following contents
FROM Quay
RUN mkdir /opt/jboss/keycloak/cache
keycloak:
build:
context: ./keycloak
dockerfile: Dockerfile
restart: unless-stopped
volumes:
- ./scripts/keycloak-import:/opt/jboss/keycloak/imports
- my_volume:/opt/jboss/keycloak/cache
- ./keycloak/standalone/configuration/standalone.xml:/opt/jboss/keycloak/standalone/configuration/standalone.xml
command: -b 0.0.0.0 -Dkeycloak.profile.feature.upload_scripts=enabled -Dkeycloak.import=${KEYCLOAK_DIR}/imports/keycloak-${APP_ENV}.json -Dkeycloak.migration.strategy=IGNORE_EXISTING
ports:
- 8080:8080
depends_on:
- db
volumes:
<cache-container name="keycloak" module="org.keycloak.keycloak-model-infinispan">
<local-cache name="realms">
<object-memory size="10000"/>
</local-cache>
<local-cache name="users">
<object-memory size="10000"/>
</local-cache>
<local-cache name="sessions">
<expiration lifespan="-1"/>
<file-store path="/opt/jboss/keycloak/cache" shared="false" preload="true" purge="false" fetch-state="true"/>
</local-cache>
<local-cache name="authenticationSessions">
<expiration lifespan="-1"/>
<file-store path="/opt/jboss/keycloak/cache" shared="false" preload="true" purge="false" fetch-state="true"/>
</local-cache>
<local-cache name="offlineSessions"/>
<local-cache name="clientSessions"/>
<local-cache name="offlineClientSessions"/>
<local-cache name="loginFailures"/>
<local-cache name="work"/>
<local-cache name="authorization">
<object-memory size="10000"/>
</local-cache>
<local-cache name="keys">
<object-memory size="1000"/>
<expiration max-idle="3600000"/>
</local-cache>
<local-cache name="actionTokens">
<object-memory size="-1"/>
<expiration interval="300000" max-idle="-1"/>
</local-cache>
</cache-container>
<cache-container name="server" default-cache="default" module="org.wildfly.clustering.server">
<local-cache name="default">
<transaction mode="BATCH"/>
</local-cache>
</cache-container>
<cache-container name="web" default-cache="passivation" module="org.wildfly.clustering.web.infinispan">
<local-cache name="passivation">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store passivation="true" purge="false"/>
</local-cache>
<local-cache name="sso">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
</local-cache>
<local-cache name="routing"/>
</cache-container>
<cache-container name="ejb" aliases="sfsb" default-cache="passivation" module="org.wildfly.clustering.ejb.infinispan">
<local-cache name="passivation">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store passivation="true" purge="false"/>
</local-cache>
</cache-container>
<cache-container name="hibernate" module="org.infinispan.hibernate-cache">
<local-cache name="entity">
<object-memory size="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<local-cache name="local-query">
<object-memory size="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<local-cache name="timestamps"/>
</cache-container>
</subsystem>
Attempt 2 - Using Keycloak 19.0.3 and configure the built-in Infinispan cache to use database persistence (Ideas taken from - jGroups/Infinispan Bedevilling: No DB Table Created (Unable to persist Infinispan internal caches as no global state enabled) - Miscellanaeous - Keycloak )
Result - Keycloak container starts without any errors, the JGROUPSPING table is created in the keycloak PostgreSQL database with 1 row (cluster_name - “ISPN”) but again the user session is lost after restarting the Keycloak container
keycloak:Quay
volumes:
- ./scripts/keycloak-import:opt/keycloak/imports
- ./keycloak/conf/cache-ispn.xml:opt/keycloak/conf/cache-ispn.xml
command: >-
start-dev
--cache=ispn
--cache-config-file=cache-ispn.xml
--db=postgres
--db-url-host=db
--db-url-database=keycloak
--db-username=$DB_USERNAME
--db-password=$DB_PASSWORD
--proxy=edge
--http-relative-path=/auth/
ports:
- 8080:8080
depends_on:
- db
volumes:
./keyclock/conf/cache-ispn.xml
<?xml version="1.0" encoding="UTF-8"?>
<jgroups>
<stack name="jdbc-ping-tcp" extends="tcp">
<JDBC_PING connection_driver="org.postgresql.Driver"
connection_url="jdbc:postgresql://127.0.0.1:5438/keycloak"
connection_username="{UserName}"
connection_password="{Password}"
initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL, cluster_name varchar(200) NOT NULL, ping_data BYTEA, constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name));"
info_writer_sleep_time="500"
remove_all_data_on_view_change="true"
stack.combine="REPLACE"
stack.position="MPING" />
</stack>
</jgroups>
<cache-container name="keycloak">
<transport lock-timeout="60000" stack="jdbc-ping-tcp"/>
<local-cache name="realms">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<local-cache name="users">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<distributed-cache name="sessions" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="authenticationSessions" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="offlineSessions" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="clientSessions" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="offlineClientSessions" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="loginFailures" owners="2">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
<local-cache name="authorization">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<replicated-cache name="work">
<expiration lifespan="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</replicated-cache>
<local-cache name="keys">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<expiration max-idle="3600000"/>
<memory max-count="1000"/>
</local-cache>
<distributed-cache name="actionTokens" owners="2">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<expiration max-idle="-1" lifespan="-1" interval="300000"/>
<memory max-count="-1"/>
<persistence>
<file-store preload="true" fetch-state="true"/>
</persistence>
</distributed-cache>
</cache-container>
I would truly appreciate it someone could provide me with some guidance
With a lot of trial and error and I think I have it working using the Keycloak 19.0.3 and file-based persistence. (Keycloak Standalone mode and the built-in Infinispan cache)
xgp
January 4, 2023, 2:42pm
6
Thank you for sharing your journey on this topic. I know this is something that a lot of people here have struggled with. Would you mind sharing your final configuration?
@xgp Yes - Happy to share the configuration I am using so far (Please note that I am still using start-dev in the commands below which should be updated to use start for production use - TODO later)
FROM Quay
RUN mkdir /opt/keycloak/cache
docker-compose.yml (keycloak service)
keycloak:
build:
context: ./keycloak
dockerfile: Dockerfile
restart: unless-stopped
volumes:
# Scripts
- ./scripts/keycloak-import:opt/keycloak/imports
# Cache
- ./keycloak/cache:opt/keycloak/cache
# Infinispan configuration
- ./keycloak/conf/cache-ispn.xml:opt/keycloak/conf/cache-ispn.xml
command: >-
start-dev
--cache=ispn
--cache-config-file=cache-ispn.xml
--db=postgres
--db-url-host=db
--db-url-database=keycloak
--db-username=$DB_USERNAME
--db-password=$DB_PASSWORD
--proxy=edge
--http-relative-path=/auth/
environment:
KEYCLOAK_ADMIN: $DB_USERNAME
KEYCLOAK_ADMIN_PASSWORD: $DB_PASSWORD
ports:
- 8080:8080
depends_on:
- db
./keycloak/conf/cache-ispan-file.xml
<?xml version="1.0" encoding="UTF-8"?>
<jgroups>
<stack name="jdbc-ping-tcp" extends="tcp">
<JDBC_PING connection_driver="org.postgresql.Driver"
connection_url="jdbc:postgresql://{IpAddress or host name}:5438/keycloak" <!-- Enter the IP address / host name here -->
connection_username="{Username}" <!-- Enter your db user name here -->
connection_password="{Password}" <!-- Enter your db password here --
initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL, cluster_name varchar(200) NOT NULL, ping_data BYTEA, constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name));"
info_writer_sleep_time="500"
remove_all_data_on_view_change="true"
stack.combine="REPLACE"
stack.position="MPING" />
</stack>
</jgroups>
<cache-container name="keycloak">
<global-state>
<persistent-location path="/opt/keycloak/cache/"/> <!-- Make sure to use the same path used in the Dockerfile & docker-compose.xml file -->
</global-state>
<transport lock-timeout="60000"/>
<!-- stack="jdbc-ping-tcp"/> -->
<local-cache name="realms">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<local-cache name="users">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<distributed-cache name="sessions" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="authenticationSessions" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="offlineSessions" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="clientSessions" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="offlineClientSessions" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<distributed-cache name="loginFailures" owners="2">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
<local-cache name="authorization">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<memory max-count="10000"/>
</local-cache>
<replicated-cache name="work">
<expiration lifespan="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</replicated-cache>
<local-cache name="keys">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<expiration max-idle="3600000"/>
<memory max-count="1000"/>
</local-cache>
<distributed-cache name="actionTokens" owners="2">
<encoding>
<key media-type="application/x-java-object"/>
<value media-type="application/x-java-object"/>
</encoding>
<expiration max-idle="-1" lifespan="-1" interval="300000"/>
<memory max-count="-1"/>
<persistence passivation="true">
<file-store shared="false" preload="true" purge="false" fetch-state="true"/>
</persistence>
</distributed-cache>
</cache-container>
1 Like
ddy
January 19, 2023, 3:58pm
8
Hi @robertcck
There is this example provided by Thomas Darimont here : keycloak-project-example/deployments/local/clusterx/haproxy-database-ispn at main · thomasdarimont/keycloak-project-example · GitHub
With database persistence, but it requires a patch on keycloak-model-infinispan since Keycloak 15.0.2
This thread might also be interesting : Keycloak.X persisting infinispan to jdbc store
We also tried the setup : external Infinispan cluster (so remote cluster) with database persistence configured in this cluster, there was a lifespan issue while persisting user sessions but it has been fixed recently and will work now : https://github.com/keycloak/keycloak/issues/10755