User storage provider timeout

I have found multiple references to the property --spi-user-storage-provider-timeout but I cannot find any documentation around it. Additionally, including it in my startup appears to have no effect as I still see custom user storage provider SPI calls timing out at 3000ms from time to time.

I was wondering if there is any information out there on configuring this timeout. It appears both client and roles have a similar timeout to the UserStorageManager, but the scope of their config object differs (user, role, client).

The source code indicates that the value should be configurable. I would appreciate any insight into how to do so.

I’m not finding references to that config in the provider config page so I’m guessing it’s no longer used.

Here’s the documentation for provider properties for reference

And here’s the reference to all supported props All provider configuration - Keycloak


Which provider are you using? Or did you write a custom provider? These props become available to the provider factory so if it’s a custom provider you’ll need to get and use that prop in code to apply a timeout.

I am using a custom user storage provider. However, it appears the issue I am seeing is due to the “wrapping” of provider calls in a timeout by the UserStorageManager class, which is an implementation of the AbstractStorageManager. This class has a default value for the timeout:

/**
     * Timeouts are used as time boundary for obtaining models from an external storage. Default value is set
     * to 3000 milliseconds and it's configurable.
     */
    private static final Long STORAGE_PROVIDER_DEFAULT_TIMEOUT = 3000L;

But mentions it is configurable. I however, have been unable to figure out the correct way to config this timeout in this class, as it does not appear to be an spi or provider, which the documentation makes a bit clearer on how to send properties in for.

I agree that it no longer appears the property I mentioned is working nor does it appear it has ever appeared in documentation, leading to my confusion.

Additionally, in that abstract manager class the constructor sets the scope of the config to “user”

    public UserStorageManager(KeycloakSession session) {
        super(session, UserStorageProviderFactory.class, UserStorageProvider.class,
                UserStorageProviderModel::new, "user");
    }

which appears to ultimately mean that the configuration of that class will come from the SystemPropertyConfigProvider and look for configuration from system properties of the form:
keycloak.user.* in this case storageProviderTimeout. Below you can see how the Abstract manager fetches that timeout from the Config.scope object with configScope=“user”.

   protected Long getStorageProviderTimeout() {
        if (storageProviderTimeout == null) {
            storageProviderTimeout = Config.scope(configScope).getLong("storageProviderTimeout", STORAGE_PROVIDER_DEFAULT_TIMEOUT);
        }
        return storageProviderTimeout;
    }

I would LOVE to understand how to correctly set this system property

It was never properly documented.
I used this property back in the time around KC18 and some dirty hack with legacy Wildfly-based Keycloaks (<17), I also created a discussion thread for this, because it was not working with Wildfly-based KC: Configurable storageProviderTimeout · keycloak/keycloak · Discussion #11989 · GitHub

Basically, it should work, because of this code:

But I don’t know, if there has been some other change around.

The not-so-configurable storageProviderTimeout .

Nice find, hah.

Yes, the configScope set in the constructor of “user” correlates to the Config.scope(configScope) call.

And the properties are CamelCased so --spi-user-my-property should map to code Config.scope("user").getLong("myProperty").

This is a runtime config so you should be able to pass the argument to the kc.sh start command. e.g.


STARTUP_ARGS="--spi-user-storage-provider-timeout=3000"
exec $KEYCLOAK_HOME/bin/kc.sh start $STARTUP_ARGS

Have you tried this? If so, what’s your startup command look like?

I have tried setting the the storage provider timeout via that manner but I always see it appear in the exception as 3000ms so it does not appear to be honored.

We leverage keycloak as a container so my startup is as part of a compose. The start command contains various args like so:

 command:
      - start
      - --optimized
      - --hostname={omitted}
      - --hostname-admin={omitted}
      - --proxy-headers=xforwarded
      - --http-enabled=true
      - --db-url={omitted}
      - --db-username={omitted}
      - --db-password={omitted}
      - --https-protocols=TLSv1.2
      - --https-certificate-key-file=/opt/certs/auth.REPLACE_SUFFIX.pem
      - --https-certificate-file=/opt/certs/auth.REPLACE_SUFFIX.pem
      - --https-trust-store-file=/opt/certs/supportedCertsKeystore.jks
      - --https-trust-store-password={omitted}
      - --log=file,console
      - --log-file=/opt/logs/keycloak.log
      - --log-level=INFO,org.keycloak.events:DEBUG
      - --log-console-format=%d{yyyy-MM-dd HH:mm:ss.SSS z} %-5p [%c] (%t) %s%e%n
      - --spi-connections-jpa-default-migration-strategy=update
      - --spi-truststore-file-file=/opt/certs/supportedCertsKeystore.jks
      - --spi-truststore-file-password={omitted}
      - --spi-x509cert-lookup-haproxy-ssl-client-cert=SSL_CLIENT_CERT
      - --spi-x509cert-lookup-haproxy-ssl-cert-chain-prefix=SSL_CLIENT_CERT_CHAIN
      - --spi-x509cert-lookup-haproxy-certificate-chain-length=1
      - --spi-user-storage-provider-timeout=5000

so it actually does look like the property works. Took a while to get a test to prove it as apparently only getUserById and getUserByUsername are subject to this timeout. I was mapping users via a certificate common name which was an attribute. FWIW this seems to only pop up when I use the common name attribute to kick off a get by the username via:

// This will do a provider get user by username and should leverage KC cache if user is present:
            user = this.itsSession.users().getUserByUsername(realmModel, username);

such that the attribute based search will benefit from the KC cache if the user exists already. This path appears to be subject to the timeout but getUserByUsername directly is either called less often in our typical use case or is not experiencing the timeout as often. I will say I do wish there was some logging of the retrieved configuration to know it is being seen as expected.