Error "Unable to find factory for AuthenticatorFactory: auth-script-based did you forget to declare it in a META-INF/services file?" after upgrade from KC15.x to 19.0.1

[tomz]

I am trying to upgrade my Docker based Nextcloud + Keycloak instance from Keycloak 15.x to 19.0.1
What I did:

1. Installed 19.0.1 image (Keycloak + dedicated DB) parallel to to 15.x
2. Old DB dump → new DB import
3. Login to 19.0.1 console is fine, all users and clients exist in new environment
4. Remove the /auth/ section from the ‘oidc_login_provider_url’ and ‘oidc_login_logout_url’ strings in Nextcloud settings
5. Delete oidc plugin cache as per Keyclock X redirect page not found · Issue #169 · pulsejet/nextcloud-oidc-login · GitHub
6. Restart Nextcloud app
7. Get error below:

keycloak-app2     | 2022-09-06 07:47:49,832 WARN  [org.keycloak.events] (executor-thread-16) type=LOGIN_ERROR, realmId=Senak, clientId=Nextcloud, userId=null, ipAddress=144.24.162.14, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code, response_type=code, redirect_uri=https://cloud.mydomain.com/apps/oidc_login/oidc, code_id=1c8ace9d-6fb0-402a-b75b-03c84f489867, response_mode=query, authSessionParentId=1c8ace9d-6fb0-402a-b75b-03c84f489867, authSessionTabId=riu6EqVgnP4
keycloak-app2     | 2022-09-06 07:47:50,712 WARN  [org.keycloak.services] (executor-thread-16) KC-SERVICES0013: Failed authentication: java.lang.RuntimeException: Unable to find factory for AuthenticatorFactory: auth-script-based did you forget to declare it in a META-INF/services file?
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.getAuthenticatorFactory(DefaultAuthenticationFlow.java:347)
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.isConditionalAuthenticator(DefaultAuthenticationFlow.java:341)
keycloak-app2     |     at java.base/java.util.function.Predicate.lambda$negate$1(Predicate.java:80)
keycloak-app2     |     at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176)
keycloak-app2     |     at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
keycloak-app2     |     at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
keycloak-app2     |     at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
keycloak-app2     |     at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
keycloak-app2     |     at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
keycloak-app2     |     at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
keycloak-app2     |     at java.base/java.util.stream.ReferencePipeline.forEachOrdered(ReferencePipeline.java:502)
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.fillListsOfExecutions(DefaultAuthenticationFlow.java:300)
keycloak-app2     |     at org.keycloak.authentication.AuthenticationSelectionResolver.addAllExecutionsFromSubflow(AuthenticationSelectionResolver.java:204)
keycloak-app2     |     at org.keycloak.authentication.AuthenticationSelectionResolver.addAllExecutionsFromSubflow(AuthenticationSelectionResolver.java:249)
keycloak-app2     |     at org.keycloak.authentication.AuthenticationSelectionResolver.createAuthenticationSelectionList(AuthenticationSelectionResolver.java:76)
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.createAuthenticationSelectionList(DefaultAuthenticationFlow.java:478)
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.processSingleFlowExecutionModel(DefaultAuthenticationFlow.java:404)
keycloak-app2     |     at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:272)
keycloak-app2     |     at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:1017)
keycloak-app2     |     at org.keycloak.authentication.AuthenticationProcessor.authenticate(AuthenticationProcessor.java:879)
keycloak-app2     |     at org.keycloak.protocol.AuthorizationEndpointBase.handleBrowserAuthenticationRequest(AuthorizationEndpointBase.java:151)
keycloak-app2     |     at org.keycloak.protocol.oidc.endpoints.AuthorizationEndpoint.buildAuthorizationCodeAuthorizationResponse(AuthorizationEndpoint.java:338)
keycloak-app2     |     at org.keycloak.protocol.oidc.endpoints.AuthorizationEndpoint.process(AuthorizationEndpoint.java:194)
keycloak-app2     |     at org.keycloak.protocol.oidc.endpoints.AuthorizationEndpoint.buildGet(AuthorizationEndpoint.java:112)
keycloak-app2     |     at jdk.internal.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
keycloak-app2     |     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
keycloak-app2     |     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
keycloak-app2     |     at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
keycloak-app2     |     at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:660)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:524)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:474)
keycloak-app2     |     at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:476)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:434)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:192)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:152)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:183)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:141)
keycloak-app2     |     at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:32)
keycloak-app2     |     at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
keycloak-app2     |     at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
keycloak-app2     |     at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
keycloak-app2     |     at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
keycloak-app2     |     at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
keycloak-app2     |     at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
keycloak-app2     |     at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:73)
keycloak-app2     |     at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:151)
keycloak-app2     |     at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.handle(VertxRequestHandler.java:82)
keycloak-app2     |     at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.handle(VertxRequestHandler.java:42)
keycloak-app2     |     at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1212)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:163)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
keycloak-app2     |     at io.quarkus.vertx.http.runtime.StaticResourcesRecorder$2.handle(StaticResourcesRecorder.java:67)
keycloak-app2     |     at io.quarkus.vertx.http.runtime.StaticResourcesRecorder$2.handle(StaticResourcesRecorder.java:55)
keycloak-app2     |     at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1212)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:163)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
keycloak-app2     |     at io.quarkus.vertx.http.runtime.VertxHttpRecorder$5.handle(VertxHttpRecorder.java:380)
keycloak-app2     |     at io.quarkus.vertx.http.runtime.VertxHttpRecorder$5.handle(VertxHttpRecorder.java:358)
keycloak-app2     |     at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1212)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:163)
keycloak-app2     |     at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
keycloak-app2     |     at org.keycloak.quarkus.runtime.integration.web.QuarkusRequestFilter.lambda$createBlockingHandler$1(QuarkusRequestFilter.java:90)
keycloak-app2     |     at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:159)
keycloak-app2     |     at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:100)
keycloak-app2     |     at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:157)
keycloak-app2     |     at io.quarkus.vertx.core.runtime.VertxCoreRecorder$13.runWith(VertxCoreRecorder.java:545)
keycloak-app2     |     at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
keycloak-app2     |     at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
keycloak-app2     |     at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
keycloak-app2     |     at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
keycloak-app2     |     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
keycloak-app2     |     at java.base/java.lang.Thread.run(Thread.java:829)
keycloak-app2     |

I can move back and forth between KC15 and 19, reverting/applying steps 4 - 7, and reliably get to the error above.
This error

“Unable to find factory for AuthenticatorFactory: auth-script-based did you forget to declare it in a META-INF/services file?”

seems to be the root cause but I have no idea what it’s trying to tell me? Can anyone point me in the right direction to fix or further troubleshoot this one?

By the way, I use the same setup (Docker+Nextcloud+OpenId-Plugin+KC19) running fine for a different installation (fresh install, not an upgrade) so this error looks to be related to the upgrade.

[gsmith]

Hello,

Not sure if this will help, but I caught these in your logs.

type=LOGIN_ERROR, realmId=Senak, clientId=Nextcloud, userId=null,

&

invalid_user_credentials

But I used SAML not openid-connect for Nextcloud 24 on Keycloak 19. perhaps check the user names being used, just a thought.

[bpedersen2]

Check if your SPI is packaged and deployed correctly for KC17+, there were quite some changes[1,2].

[1] Migrating to Quarkus distribution - Keycloak
[2] Server Developer Guide

[tomz]

I noticed that one too, but think it’s a red herring because the error happens before I even have a chance to enter a user account, hence userId ought to be empty?

[tomz]

I checked the links you provided but unless I misunderstood this concept I don’t think I use any specific SPI. I use the default docker images for 15.x and Quay without adding any other modules?
But, now that I write this, could a Script Mapper lead to a problem like this? I got one simple script to create Mattermost IDs on the fly

[bpedersen2]

Double-check that your scirpts are deployed correctly, as deployment changed quite a bit in quarkus, see [1].

[1] Server Developer Guide

[tomz]

You hit the issue, they are not!
Top image is my old setup, bottom the migrated one:

Clearly the script mapper did not migrate. And as I type this note I remembered that on v15 I had to add - "-Dkeycloak.profile.feature.upload_scripts=enabled" as environment variable, and it seems the same exists for v19. There is a script feature listed here: Enabling and disabling features - Keycloak

I did enable it and it shows as enabled during launch:
2022-09-11 11:55:12,771 INFO [org.keycloak.common.Profile] (build-31) Preview feature enabled: scripts

But there’s still is no “Script mapper” type to pick from when selecting “Add mapper” above.

[mbonn]

Javascript mappers have to be deployed as a jar file:
https://www.keycloak.org/docs/latest/server_development/index.html#_script_providers

The “script” feature has to be enabled, too.
(The “upload_scripts” feature is removed, so scripts are no more stored as text in the database.)

After that, your scripts will appear as a new mapper type in the mappers dialogs of the admin console.

[tomz]

Boy oh boy, handling scripts seems to be a lot more complicated than it used it to be.
Anyway, here’s what I think the documentation tells me to do:

• 1 Created 2 files, the actual script file and keycloak-scripts, with the later in the META-INF subdir, #1
  I only added the mapper section to keycloak-scripts.json because I only have a mapper script.

• Zipped it up and renamed the zip to jar, #2 shows the content and structure of the jar/zip file

• Put the jar file into the keycloak/providers folder and it shows:

docker exec -it keycloak-app2 /bin/bash
bash-4.4$ ls /opt/keycloak/providers
mattermostMapper.jar

• Restarted Keycloak docker container, not sure if the jar file has been picked up, but at least I don’t get an error:

keycloak-app2     | Updating the configuration and installing your custom providers, if any. Please wait.
keycloak-app2     | 2022-09-12 07:49:33,493 INFO  [org.keycloak.common.Profile] (build-3) Preview feature enabled: scripts

Should that be it? There’s still no corresponding mapper in the dedicated scopes section. Are there any other places I could check if the script was accepted or look for potential errors?

Edit:
With the stuff I learned above, the error message now starts to make sense:

Unable to find factory for AuthenticatorFactory: auth-script-based did you forget to declare it in a META-INF/services file?

But doesn’t that also mean the mapper has been properly migrated and should somehow already show in the admin ui? It’s obviously looking for the script during logon which wasn’t there until now.

However, given that this is turning out way more complicated than expected I am considering getting rid of the script and to provide the id mattermost needs as a custom user property. Tried but still get the error, so somehow the script setting is lurking somewhere. I may have to recreate the Client I guess…

[tomz]

Couldn’t work this out I am afraid, the “hidden script” was blocking all clients, including those which didn’t make use of the script.
Had to recreate all clients and set up of a manual process to replace the script.

[mbonn]

What Java Version are you running?

[tomz]

Here’s what java --version tells me, I assume it’s the one packaged with the docker image?

sudo docker exec keycloak-app java --version                                           
openjdk 11.0.16 2022-07-19 LTS
OpenJDK Runtime Environment (Red_Hat-11.0.16.0.8-1.el8_6) (build 11.0.16+8-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-11.0.16.0.8-1.el8_6) (build 11.0.16+8-LTS, mixed mode, sharing)

[mbonn]

I asked because the nashorn scripting engine (which is needed to execute script mappers in keycloak) is removed from JDK 17. But on JDK 11 it should be there… Also make sure not to forget bin/kc.[sh|bat] build…

[tomz]

Thanks for helping along, but I have given up on it. I created new clients which don’t seem to be linked to the script causing the Unable to find factory for AuthenticatorFactory error.
So everything is working now, and for that one client which required the script I manually create userids. Not ideal, but I can’t spend more time on this.