Keycloak-x docker startup-scripts not being executed

cedricguindon · March 30, 2021, 12:05pm

Hello

I’m currently testing the keycloak-x application and i’m trying to run a bash script as soon as keycloak-x is started but with no luck and i’m wondering if anyone got it to work. Here’s my entire process.

1: Building the image with the following:

docker build . --tag keycloak_test_startup:latest

here’s my Dockerfile

FROM quay. io/keycloak/keycloak-x:latest
COPY providers /opt/jboss/keycloak/providers/
COPY startup-scripts /opt/jboss/startup-scripts/
WORKDIR /opt/jboss/keycloak

# Creating a self-signed for testing purposes
RUN keytool -genkeypair -storepass password -storetype PKCS12 -keyalg RSA -keysize 2048 -dname “CN=server” -alias server -ext “SAN:c=DNS:localhost,IP:127.0.0.1” -keystore conf/server.keystore

# Run the config command to install custom providers
RUN ./bin/kc.sh config

Here’s the content of startup-scripts:

cedric@cedric-work-pc:~/Documents/test$ ls -la ./startup-scripts/
total 96
drwxrwxrwx 2 cedric cedric 4096 Mar 19 14:56 .
drwxrwxr-x 5 cedric cedric 4096 Mar 30 07:46 …
-rwxrwxrwx 1 cedric cedric 85341 Mar 10 08:23 my_realm.json
-rwxrwxrwx 1 cedric cedric 386 Mar 19 13:22 testingfile.sh

testingfile.sh:

#!/bin/bash

echo “inside testingfile”;

for i in {1…10}; do
echo “test: $i”;
sleep 5s
done

echo “after loop”;

/opt/jboss/keycloak/bin/kcadm.sh config credentials --server "http://"localhost:8080/ --realm master --user admin --client admin-cli --password “PASSWORD”;
/opt/jboss/keycloak/bin/kcadm.sh create realms -s realm=demorealm -s enabled=true;
echo “after create realms”;

So far so good, the image get’s built correctly.

Starting up:

docker run --name keycloak_test_startup -p 8080:8080 --net keycloak-network -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=“PASSWORD” keycloak_test_startup:latest --auto-config --db=postgres -Dkc.db.url.host=“INTERNAL DB ADDRESS” --db-username=keycloak --db-password=“DB PASSWORD” --http-enabled=true

Here’s the output when starting up:

Updating the configuration and installing your custom providers, if any. Please wait.
Server configuration updated and persisted. Run the following command to review the configuration:

kc.sh show-config

2021-03-30 11:52:38,019 WARN [org.key.qua.KeycloakRecorder] (main) New property [kc.db.url.host] set with value [“INTERNAL DB ADDRESS”] in [SysPropConfigSource]. This property is not persisted into the server image.
2021-03-30 11:52:38,025 WARN [org.key.qua.KeycloakRecorder] (main) Please, run the ‘config’ command if you want to persist the new configuration into the server image:

kc.sh config --db=postgres --db-username=keycloak --db-password=“DB PASSWORD” --http-enabled=true --db-url-host=“INTERNAL DB ADDRESS”

2021-03-30 11:52:39,325 WARN [io.qua.run.ConfigChangeRecorder] (main) Build time property cannot be changed at runtime. quarkus.package.type was fast-jar at build time and is now mutable-jar
2021-03-30 11:52:40,287 INFO [org.key.url.DefaultHostnameProviderFactory] (main) Frontend: , Admin: , Backend:
2021-03-30 11:52:40,451 INFO [org.key.pro.qua.QuarkusCacheManagerProvider] (main) Loading cluster configuration from /opt/jboss/keycloak/bin/…/conf/cluster-default.xml
2021-03-30 11:52:40,898 INFO [org.inf.CONTAINER] (main) ISPN000128: Infinispan version: Infinispan ‘Corona Extra’ 11.0.4.Final
2021-03-30 11:52:41,043 INFO [org.inf.CLUSTER] (main) ISPN000078: Starting JGroups channel ISPN
2021-03-30 11:52:41,043 INFO [org.inf.CLUSTER] (main) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2021-03-30 11:52:41,133 WARN [org.jgr.pro.UDP] (main) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2021-03-30 11:52:41,133 WARN [org.jgr.pro.UDP] (main) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
2021-03-30 11:52:41,133 WARN [org.jgr.pro.UDP] (main) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2021-03-30 11:52:41,134 WARN [org.jgr.pro.UDP] (main) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
2021-03-30 11:52:43,144 INFO [org.jgr.pro.pbc.GMS] (main) af4fc943196a-6956: no members discovered after 2003 ms: creating cluster as coordinator
2021-03-30 11:52:43,183 INFO [org.inf.CLUSTER] (main) ISPN000094: Received new cluster view for channel ISPN: [af4fc943196a-6956|0] (1) [af4fc943196a-6956]
2021-03-30 11:52:43,201 INFO [org.inf.CLUSTER] (main) ISPN000079: Channel ISPN local address is af4fc943196a-6956, physical addresses are [172.18.0.3:48112]
2021-03-30 11:52:43,669 INFO [org.key.con.inf.DefaultInfinispanConnectionProviderFactory] (main) Node name: af4fc943196a-6956, Site name: null
2021-03-30 11:52:43,834 INFO [org.key.con.jpa.QuarkusJpaConnectionProviderFactory] (main) Database info: {databaseUrl=jdbc:postgresql://“INTERNAL DB ADDRESS”:5432/keycloak?allowEncodingChanges=false&ApplicationName=PostgreSQL+JDBC+Driver&autosave=never&binaryTransfer=true&binaryTransferDisable=&binaryTransferEnable=&cancelSignalTimeout=10&cleanupSavepoints=false&connectTimeout=10&databaseMetadataCacheFields=65536&databaseMetadataCacheFieldsMiB=5&defaultRowFetchSize=0&disableColumnSanitiser=false&escapeSyntaxCallMode=select&gssEncMode=allow&gsslib=auto&hideUnprivilegedObjects=false&hostRecheckSeconds=10&jaasLogin=true&loadBalanceHosts=false&loginTimeout=0&logServerErrorDetail=true&logUnclosedConnections=false&preferQueryMode=extended&preparedStatementCacheQueries=256&preparedStatementCacheSizeMiB=5&prepareThreshold=5&readOnly=false&readOnlyMode=transaction&receiveBufferSize=-1&reWriteBatchedInserts=false&sendBufferSize=-1&socketTimeout=0&sspiServiceClass=POSTGRES&targetServerType=any&tcpKeepAlive=false&unknownLength=2147483647&useSpnego=false&xmlFactoryFactory=, databaseUser=keycloak, databaseProduct=PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1), databaseDriver=PostgreSQL JDBC Driver 42.2.18}
2021-03-30 11:52:44,660 ERROR [org.key.services] (main) KC-SERVICES0010: Failed to add user ‘admin’ to realm ‘master’: user with username exists
2021-03-30 11:52:45,216 INFO [io.quarkus] (main) Keycloak 12.0.4 on JVM (powered by Quarkus 1.10.0.CR1) started in 7.639s. Listening on: "http://"0.0.0.0:8080 and "https://"0.0.0.0:8443
2021-03-30 11:52:45,217 INFO [io.quarkus] (main) Profile prod activated.
2021-03-30 11:52:45,217 INFO [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mysql, jdbc-postgresql, keycloak, mutiny, narayana-jta, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-health, smallrye-metrics, vertx, vertx-web]

Sorry for the output mess but i’m not sure how i can make it better.

As it look like, the image started, connected to the DB, admin account was previously created when starting up before, nothing seem to be out of the ordinary at first glance.

I connected to the container to make sure the bash script was indeed in there:

bash-4.4$ ls -la
total 100
drwxr-xr-x 2 root root 4096 Mar 30 11:41 .
drwxrwxr-x 1 jboss root 4096 Mar 30 11:41 …
-rwxrwxrwx 1 root root 85341 Mar 10 13:23 my_realm.json
-rwxrwxrwx 1 root root 386 Mar 19 17:22 testingfile.sh
bash-4.4$ pwd
/opt/jboss/startup-scripts

It is present.

But the file is never executed, no echo in the logs and the realm does not get created, if i login the container and execute the script the realm gets created without any problem.

I’m not sure what i can try anymore, any help would be appreciated.

Thank you!

cedricguindon · March 31, 2021, 2:49pm

I did some more research and now i’m wondering if the startup-scripts method is even possible in keycloak-x.

Regular keycloak : docker-entrypoint.sh contains the /opt/jboss/tools/autorun.sh script, which by itself contains the script execution fro the startup-scripts path
Keycloak-x: The docker entrypoint has nothing about the startup-scripts (it’s also much simple if form of commands)

Anybody could confirm?

cedricguindon · March 31, 2021, 6:46pm

Here’s another update

I finally got the startup-scripts to load with keycloak-x.

But now it seems that the kcadm.sh script can’t connect to localhost, here’s the specific output when doing the docker run command:

Ignoring file in /opt/jboss/startup-scripts (not *.cli or executable): /opt/jboss/startup-scripts/my_realm.json
Executing: /opt/jboss/startup-scripts/testingfile.sh
inside testingfile
test: 1
test: 2
test: 3
test: 4
test: 5
after loop
Logging into "http://"localhost:8080/ as user admin of realm master
Failed to send request - Connect to localhost:8080 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
after config credentials
No server specified. Use --server, or ‘kcadm.sh config credentials or connection’.
after create realms

I made some modification to the testfile.sh

It only loops 5 time instead of 10, but you can see the echo works.
There’s no " around the http://, that’s only to prevent the spam from this forum
The password is in single quote instead of double quote

I tried the container IP instead of localhost, same error.

cedricguindon · April 1, 2021, 1:54pm

Update

I finally figured it out, my script is now connecting to the keycloak-x service by simply adding the & so my script is executed in the background, and after the sleep time, the service is started and create realm works.

I was able to fix it.

This brings me to another point, keycloak-x should implement the startup-scripts method, my guess once keycloak-x gets to an official build this feature will be added. Something to add to the feature list.

famod · June 2, 2022, 9:22pm

Hi @cedricguindon!

So it seems that startup-scripts still hasn’t made it to Keycloak.X (yet?), which is very unfortunate.

Can you share your workaround? Thanks!

cedricguindon · June 3, 2022, 12:27pm

@famod
I switched to keycloak 18 instead of keycloak.X, startup-scripts is also not there with this one.

The workaround i’m using is the following:

My Dockerfile has a custom entrypoint, init.sh

# Create new entrypoint.
COPY ./docker/init.sh /
RUN chmod -R 554 /init.sh

ENTRYPOINT [ "/init.sh", "start" ]

As you can see with the above, i’m injecting the init.sh file and using that file as my new ENTRYPOINT.

The magic is in my main() function in init.sh

function main() {
    # Parameters
    local keycloak_cmd_arguments=("$@")
    local spi_arguments="--spi-login-freemarker-enabled=false --spi-login-csps-freemarker-login-messages-enabled=true --spi-eventsListener-csps-login-event-listener-enabled=true --spi-eventsListener-csps-logout-event-listener-enabled=true --spi-theme-welcome-theme=csps"

    wait_for_database_service_availability
    create_database_and_credentials

    /opt/startup-scripts/keycloak_startup_script.sh &

    # Launch base container entrypoint with container's runtime cmd arguments..."
    case "${ENVIRONMENT_NAME}" in
      dev-local)
        /opt/keycloak/bin/kc.sh start-dev "${spi_arguments}" --http-relative-path /auth
        ;;
      *)
        /opt/keycloak/bin/kc.sh "${keycloak_cmd_arguments[@]}" "${spi_arguments}"
        ;;
    esac
}

main "$@"

What’s interesting in the above is the line:

/opt/startup-scripts/keycloak_startup_script.sh &

This line will simply call my original script, previously copied from the Dockerfile, the one i used to have in the startup_script folder but with the “&” argument so it can run in the background.

At this point the keycloak service is not started yet, but it will start within the “case” depending in which environment we’re running it.

My startup script has the following:

function wait_for_keycloak() {
  local -r MAX_WAIT=60
  local curl_request
  local host_url="http://localhost:8080/auth"
  local wait_time

  curl_request="curl -I -f -s ${host_url}"
  wait_time=0

  # Waiting for the application to return a 200 status code.
  until ${curl_request}; do
    if [[ ${wait_time} -ge ${MAX_WAIT} ]]; then
      logger::error "The application service did not start within ${MAX_WAIT} seconds. Aborting."
      exit 1
    else
      logger::info "Waiting (${wait_time}/${MAX_WAIT}) ..."
      sleep 1
      ((++wait_time))
    fi
  done

  logger::info "${host_url} is now up and running."
}

# Waiting for Keycloak to start before proceeding with the configurations.
wait_for_keycloak

# Keycloak is running.

logger::title "Calling configure_keycloak"
# shellcheck disable=SC2154 # CUSTOM_SCRIPTS_DIR is defined in Dockerfile.
"${CUSTOM_SCRIPTS_DIR}"/configure_keycloak.sh
logger::title "End of configure_keycloak"

As you can see in the above, once i execute the startup-script in the background, i’m actively checking if the keycloak service is up and running during a 60 seconds timeframe, if it is running i’m calling in my configure script, if it’s not running within 60 seconds there’s a problem somewhere.

To conclude, i’m simply running a check in the background until the keycloak service is running, and once it is im calling my configure script.

There might be better solutions but this approach works well.
The cons of this is if the keycloak service takes a long time to start, more than 60 seconds, then the configure will never be called but i have yet to find a better way of checking if the service is started. There’s the /health route but you still have to ping that route until it works so it’s the same thing as pinging /auth (/auth is not default in keycloak 18, keep that in mind). You could set it up in a way that it’s indefinatly waiting but if something happens the loop will never end.

Hope it helps

famod · June 15, 2022, 12:18pm

Thanks @cedricguindon!

I came up with a very simple solution myself: Support/replacement for startup-scripts? · Discussion #12058 · keycloak/keycloak · GitHub

Topic		Replies	Views
Keycloak-x docker startup-scripts not being executed Configuring the server	0	202	March 19, 2021
Keycloak startup-scripts in docker container Configuring the server	3	6585	May 7, 2020
Execute script on keycloak restart Configuring the server	2	244	December 12, 2023
Run script after server startup	9	5729	August 12, 2022
Running admin CLI to create clients in Docker Configuring the server	0	370	November 12, 2020

Keycloak-x docker startup-scripts not being executed

Related Topics