Keycloak Custom SPI Deployment with external Jar

cmlonder · April 13, 2020, 10:26am

I have a SPI implementation which is:

Custom SPI, directly implemented via SPI interface
has an external jar which does not exist in JBoss base

EAR deployment: I tried EAR deployment and deploy my .ear to /standalone/deployments. This solves the external jar problem which is bundled within EAR’s lib folder. But now SPI is not initializing (which I saw when I debug) and also I get an exception when I trigger the SPI:

11:34:02,185 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 9.0.2 (WildFly Core 10.0.3.Final) started in 12070ms - Started 732 of 1031 services (613 services are lazy, passive or on-demand)
    11:34:18,209 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-1) Uncaught server error: java.lang.NullPointerException
            at org.keycloak.keycloak-services@9.0.2//org.keycloak.services.DefaultKeycloakSessionFactory.getAllProviderIds(DefaultKeycloakSessionFactory.java:362)

Then I thought maybe keycloak is not able to import extended SPI via standalone/deployment deployment, which is also mentioned here if you develop custom SPI keycloak suggests (or requires?) module deployment.

Then I tried module deployment; now I can see that my custom SPI is initializing, but now keycloak can not find my external JAR.

    13:17:05,682 FATAL [org.keycloak.services] (ServerService Thread Pool -- 65) java.lang.RuntimeException: org.jboss.modules.ModuleNotFoundException: com.orbitz.consul

As a solution, I found somewhere that I can put my dependent jar and all its dependent jar’s to ${KEYCLOAK_HOME}\modules\system\layers\keycloak but I don’t want to install my external JAR and its all dependencies manually to keycloak’s base (maybe automatically somehow?). Any solution?

Script to deploy as a module:

./jboss-cli.bat --command="module add --name=de.easy.one.bouncer.spi.registry --resources=target/registry-spi-1.0.1-SNAPSHOT.jar --dependencies=org.keycloak.keycloak-core,org.keycloak.keycloak-services,org.keycloak.keycloak-server-spi,org.keycloak.keycloak-server-spi-private,javax.api,javax.ws.rs.api,com.fasterxml.jackson.core.jackson-core,com.fasterxml.jackson.core.jackson-databind,com.fasterxml.jackson.core.jackson-annotations,org.jboss.logging,com.orbitz.consul"

And added it to standalone.xml as follows:

    <provider>module:de.easy.one.bouncer.spi.registry</provider>

META-INF/services

file name --> file content

org.keycloak.provider.Spi --> de.easy.one.bouncer.spi.registry.spi.RegistryProviderSpi

de.easy.one.bouncer.spi.registry.spi.RegistryProviderFactory --> de.easy.one.bouncer.spi.registry.consul.ConsulRegistryProviderFactory

my pom.xml

    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <scope>provided</scope>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-server-spi</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-server-spi-private</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-services</artifactId>
            <scope>provided</scope>
        </dependency>

        <!-- this is not provided and I want to use as external jar -->
        <dependency>
            <groupId>com.orbitz.consul</groupId>
            <artifactId>consul-client</artifactId>
            <version>1.4.2</version>
        </dependency>
    </dependencies>

bitrecycling · August 2, 2020, 6:54pm

Keycloak automatically deploys jars (given they have valid META-INF) as the “correct type” of module:

for a user federation SPI , the META-INF/services must contain a file named org.keycloak.storage.UserStorageProviderFactory
this file must contain a line with the fully qualified class name to the UserStorageProviderFactory implementation, e.g. com.mycompany.keycloak.custom.spi.MyCompanyUserStorageProviderFactory

If that is true (in that mentioned “external jar”), then you can deploy it by copying to to path-to-keycloak/standalone/deployments

alvaroarenashel · December 23, 2020, 6:55am

Thank you for your reply, @bitrecycling

I am able to deploy my custom authenticator and it worked fine until I had to use a third party library which is not provided (e.g. keycloak-service)

How should the third party library be deployed?
How is the correct way to reference the library from my code/jar?

martinleim · December 28, 2020, 12:49pm

Hi @cmlonder,

as far as I see it, you have three possibilities:

Install the external JAR as a module. Deploy your extension either as a module or as a JAR and let it reference that module as well as the Kecloak modules as described in https://www.keycloak.org/docs/latest/server_development/#registering-provider-implementations.
Package your extension as an EAR or WAR. Add the consul-client dependency to the EAR/WAR and add add module dependencies to the other Keycloak modules (via jboss-deployment-structure.xml or manifest).
Add the classes from the consul-client dependency to your JAR (e.g. using the maven-shade-plugin/). Use the same module reference as in 1, except for the consul-client.

Personally, I would go for the second, but all three should work.

tania · May 17, 2023, 1:19pm

Hi @martinleim,
I am developing a custom provider, which uses an external library for accessing an existing database (mysql). I am using keycloak server 21.1.0.

Using shadowjar i have included into the jar the external library. When i try to run the keycloak using docker i am getting the following error:

Updating the configuration and installing your custom providers, if any. Please wait.
The DelayedHandler was closed before any children handlers were configured. Messages will be written to stderr.
2023-05-17 12:34:59,907 DEBUG [org.jboss.logging] (main) Logging Provider: org.jboss.logging.JBossLogManagerProvider

2023-05-17 12:35:00,215 DEBUG [io.quarkus.bootstrap.classloading.QuarkusClassLoader] (main) Adding elements io.quarkus.bootstrap.classloading.PathTreeClassPathElement[org.keycloak:keycloak-quarkus-server / runtime=true resources=null] to QuarkusClassLoader Augmentation Class Loader: PROD

The structure for the provider is
creating: com/xxxx/
creating: com/xxxx/yyy/
creating: com/xxxx/yyy/zzz/
creating: com/xxxx/yyy/zzz/keycloak/
creating: com/xxxx/yyy/zzz/keycloak/provider/
Whereas for the libraries are:
creating: com/xxxx/yyy/qqq/
creating: com/xxxx/yyy/qqq/wwww/

Is there any recommendation on this issue?

weltonrodrigo · May 17, 2023, 1:52pm

For que quarkus-based keycloak (17+), you can include all dependencies in the provider folder as separate jars.

tania · May 17, 2023, 2:20pm

@weltonrodrigo Is there any example on how i can do that, or any resource? It will be really valuable. I use gradle

weltonrodrigo · May 18, 2023, 12:30pm

See Uncaught server error: java.lang.NoClassDefFoundError implementing a custom provider for an external mysql database - #3 by tania

tania · June 6, 2023, 7:41am

Hello all, how could i import a custom provider during deployment (and not the admin ui), giving the name and the properties required by the custom provider?
In the dockerfile i have put the appropriate jar into the /opt/keycloak/providers/.

Thank you in advance.

toytanik · December 17, 2023, 10:26pm

Any update here ? I am getting this error : Uncaught server error: java.lang.NoClassDefFoundError: com/messagebird/MessageBirdService.I created this file based on tania’s comment not working. help help help, messagebird 6.1.2 and keycloak 22.0.5
my gradle :

plugins {
    id 'java'
    id 'com.github.johnrengelman.shadow' version '7.1.2' (gradle, 7.4)
}

group 'nl.company'
version = 'DUMMY_VERSION_THAT_ALWAYS_GETS_REPLACED_BY_CI'

tasks.register('setVersionToCommitSha') {
    doLast {
        String s = buildFile.getText().replaceFirst("version = '$version'", "version = '" + inputCommitSha + "'")
        buildFile.setText(s)
    }
}

sourceCompatibility = JavaVersion.VERSION_17
repositories {
    mavenCentral()
}

configurations {
    bundleLib
}

dependencies {
    implementation "org.keycloak:keycloak-server-spi-private:${keycloak}"
    implementation "org.keycloak:keycloak-server-spi:${keycloak}"
    implementation "org.keycloak:keycloak-services:${keycloak}"
    implementation "com.messagebird:messagebird-api:${messagebird}"

    testImplementation "org.junit.jupiter:junit-jupiter-api:${junit}"
    testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${junit}"

}

jar {
    from {
        configurations.bundleLib.collect { it.isDirectory() ? it : zipTree(it) }
    }
    manifest {
        attributes(
                'Dependencies': "org.keycloak.keycloak-services"
        )
    }
}

shadowJar {

    dependencies {
        include(dependency("org.keycloak:keycloak-services:$keycloak")) // keycloak 22.0.5
        include(dependency("com.messagebird:messagebird-api:${messagebird}"))// messagebird 6.1.2
        include(dependency("org.keycloak:keycloak-server-spi-private:${keycloak}"))
        include(dependency("org.keycloak:keycloak-server-spi:${keycloak}"))
        include(dependency("org.keycloak:keycloak-services:${keycloak}"))
    }
}

Topic		Replies	Views
Keycloak Custom SPI Deployment external Jar in modules not found	0	549	December 17, 2020
Not able to register a custom SPI Extending the server	22	8523	December 1, 2022
SPI JAR Deployment and jboss-deployment-structure.xml Extending the server	2	1010	January 12, 2022
Deploying a custom SPI: using Keycloak deployer vs. JBoss API	1	431	May 20, 2021
Extending Keycloak with jar file without SPI Extending the server	0	792	November 16, 2020

Keycloak Custom SPI Deployment with external Jar

Related Topics