Um, I don’t think so. Otherwise, x509 authentication would not work at all. x509 authentication works fine unless keycloak is behind an ingress controller on Kubernetes.
In my case, keycloak is behind only Nginx reverse proxy.
I added these in Nginx config hope it helps you:
I intend to pack keycloak in a container image but until now it still works according to my wishes so I don’t want to do that for the time being.
From Keycloak’s point of view, there is nothing special about requests coming from mutual-ssl clients. The only difference is that those requests have a Ssl-client-certificate HTTP header.
So, you still need to configure keycloak to extract that certificate. It seems that part is ok.
With the certificate in it’s hands, keycloak can now be configured to use it for use authentication.
See Server Administration Guide on how to configure x509 authentication (it can be optional, with a user password form, or any other supported by keycloak, as a fallback).
1 Like
This from Keycloak guides:
I passed the certificate validation and was redirected to my app.
But I don’t understand how to save or use the access token from the returned keycloak to call and use the service from my back end.
Do you understand it well? Could you please let me know?
Thank you
Sorry, I’ve never seen that error message before
Hi.
With the x509 authentication configured on Keycloak, now you can do the openid-connect authentication flow.
To become more familiar with it, I suppose you could take a look at an introduction to OIDC like this: https://janrain-education-center.knowledgeowl.com/home/an-introduction-to-oauth-20-and-openid-connect
Basically, you should use an OIDC library in your app, configure it with the keycloak server’s url, client_id, client_secret.
The library will take controle of the authentication flow and return to you an authenticated user. You can now use the access_token to talk to your backend API, etc.
I don’t recommend you try implement openid-connect flow yourself. Use a library for it and see how you can extract the access_token from it (in case of javascript applications) or how to proceed after authentication (for something like a spring application).
You could take a look at some guide with a google search like: openid-connect x, where x is your stack, like openid-connect nodejs or openid-connect springboot.
1 Like
@weltonrodrigo okay, back on this today. question: I believe I am using the Subject Certificate Alternative Name that contains my email, which is the Identity Source for my Keycloak instance:
I wonder if it is possible that the ingress is preventing this header field from reaching keycloak?
@weltonrodrigo here this shows the truststore is loaded correctly but the http header is empty:
[32m21:51:56,387 DEBUG [org.keycloak.services.x509.NginxProxySslClientCertificateLookup] (default task-1) Loading Keycloak truststore ...
^[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.services.x509.NginxProxySslClientCertificateLookup] (default task-1) Keycloak truststore loaded for NGINX x509cert-lookup provider.
^[[0m^[[33m21:51:56,387 WARN [org.keycloak.services.x509.AbstractClientCertificateFromHttpHeadersLookup] (default task-1) HTTP header "" is empty
^[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.services] (default task-1) [X509ClientCertificateAuthenticator:authenticate] x509 client certificate is not available for mutual SSL.
Try those settings with keycloak 18:
If that doens’t work, deploy daime/http-dump image and point your ingress to that just to check if the mutual-ssl is really ok at the ingress level. You should see the correct headers being set, with certificate at the Ssl-Client-Cert header (that would be ssl-client-cert for keycloak at the header name config).
If that is already ok at your side, I’d begin investigating the settings for the actual authenticator. I followed the x509 guide and it worked like a charm once the certificate was at the header and the keystore was correct.
@weltonrodrigo thanks for responding so quickly → really appreciate your efforts no this! Just to reiterate…x509 authentication is working perfectly IF I am not accessing Keycloak via ingress on Kubernetes, so this is definitely and only an issue with configuring the ingress element of the Keycloak kubernetes deployment.
Specifically, the truststore element is correct (otherwise x509 authentication would not work at all), both the truststore and the Nginx x509cert-lookup provider are loaded/registered correctly:
[[0m^[[32m09:11:02,226 DEBUG [org.keycloak.services.DefaultKeycloakSessionFactory] (ServerService Thread Pool -- 63) Loaded SPI x509cert-lookup (provider = nginx)
^[[0m^[[32m09:11:02,227 DEBUG [org.keycloak.models.sessions.infinispan.InfinispanStickySessionEncoderProviderFactory] (ServerService Thread Pool -- 63) Should attach route to the sticky session cookie: true
^[[0m^[[32m09:11:02,233 DEBUG [org.keycloak.truststore.FileTruststoreProviderFactory] (ServerService Thread Pool -- 63) Trusted root CA found in trustore : alias : caroot | Subject DN : CN=TrustForge Root CA-1
^[[0m^[[32m09:11:02,240 DEBUG [org.keycloak.truststore.FileTruststoreProviderFactory] (ServerService Thread Pool -- 63) File truststore provider initialized: /opt/bitnami/keycloak/certs/keycloak.truststore.jks
^[
…and the NginxProxySslClientCertificateLookup is loaded correctly and the lookup on the incoming request is attempted as shown below. What is also shown below is that, at a minimum, the certificate headers are not making it through. I believe the certificate appears to be making it through; otherwise, there would be a 400 error at the Nginx controller.
[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.services.x509.NginxProxySslClientCertificateLookup] (default task-1) Loading Keycloak truststore ...
^[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.services.x509.NginxProxySslClientCertificateLookup] (default task-1) Keycloak truststore loaded for NGINX x509cert-lookup provider.
^[[0m^[[33m21:51:56,387 WARN [org.keycloak.services.x509.AbstractClientCertificateFromHttpHeadersLookup] (default task-1) HTTP header "" is empty
^[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.services] (default task-1) [X509ClientCertificateAuthenticator:authenticate] x509 client certificate is not available for mutual SSL.
^[[0m^[[32m21:51:56,387 DEBUG [org.keycloak.authentication.DefaultAuthenticationFlow] (default task-1) authenticator ATTEMPTED: auth-x509-client-username-form
So I am thinking there is an error in the keycloak values.yaml file in the nginx ingress annotations section and that’s where I am focusing. If you have an example of accessing ingress-enabled keycloak on kubernetes, that would be totally awesome. I’ve scoured the internet and have yet to find an example of someone doing this, which I find puzzling since the bitnami keycloak kubernetes deployment has an ingress section.
If you have the actual, deployed keycloak ingress output via the following kubectl command…
kubectl edit -o yaml ingress keycloak
…that’s what I really need. I can edit my deployment in place to match what you have.
–John
@weltonrodrigo this is my kubernetes ingress config w/ the latest suggestion (I think) from you:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
meta.helm.sh/release-name: keycloak
meta.helm.sh/release-namespace: demo
nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-User,X-Auth-Request-Email
nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
nginx.ingress.kubernetes.io/auth-tls-secret: demo/keycloak-ca-secret
nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Accept-Encoding "";
proxy_set_header Ssl-Client-Cert $ssl_client_cert;
proxy_set_header proxy-ssl-client-subject-dn $ssl_client_s_dn;
proxy_set_header proxy-ssl-client-issuer-dn $ssl_client_i_dn;
proxy_set_header proxy-ssl-client-verify $ssl_client_verify;
nginx.ingress.kubernetes.io/enable-rewrite-log: "true"
creationTimestamp: "2022-07-05T15:30:51Z"
generation: 3
Still getting this error:
[[0m^[[32m09:43:40,204 DEBUG [org.keycloak.authentication.DefaultAuthenticationFlow] (default task-1) invoke authenticator.authenticate: auth-x509-client-username-form
^[[0m^[[33m09:43:40,204 WARN [org.keycloak.services.x509.AbstractClientCertificateFromHttpHeadersLookup] (default task-1) HTTP header "" is empty
^[[0m^[[32m09:43:40,204 DEBUG [org.keycloak.services] (default task-1) [X509ClientCertificateAuthenticator:authenticate] x509 client certificate is not available for mutual SSL.
@hokiegeek2 hi - does your demo/keycloak-ca-secret contain ca.crt as per
Client Certificate Authentication
This secret must have a file named
ca.crtcontaining the full Certificate Authority chainca.crtthat is enabled to authenticate against this Ingress.
hi @mikhust thank you for the follow-up! The secret contains ca.crt file with the root ca cert but not the intermediate ca, so maybe therein lies the problem
Hi, have you managed to get this to work?
After your first message, I became interested in this configuration out of curiosity. I managed to get it working after some pain.
It’s actually simple, but don’t trust the fact that you can get it to work when accessing keycloak directly because there is actually no standard way of passing mutual-tls information upstream (from the reverse proxy to the application - keycloak in this case) and keycloak components are configured out of the box to do that, but must be configured when a reverse proxy is present.
So, you’ll need first to convince nginx to do it in a way that keycloak accepts.
I cannot recommend strongly enough that you should use an HTTP dump pod instead of keycloak to see if the headers are being set correctly before going forward into keycloak configuration.
I don’t believe you need the nginx.ingress.kubernetes.io/configuration-snippet, just the cacert secret configuration should be enough to activate mutual-tls on nginx, the default header storing the certificate is ssl-client-cert (capitalized to Ssl-Client-Cert as headers must be by following the HTTP standard).
After that, having the cert in the right header, you can configure keycloak as discussed to get it from that header (the syntax of that configuration is unfortunately pretty strange, but that should get you going.
Please be aware that it’s required that the truststore be correct with the intermediate certificates (which doesn’t matter when connecting directly via keycloak, because it can extract then from the headers - which nginx doesn’t set).
Good luck.
@weltonrodrigo thanks a bunch for the follow-up! I’ve been working on other things for a few days, so this is not resolved yet. i think @mikhust was on to something, because once we disabled TLS termination, I was getting nginx errors indicating a problem w/ the CA. I think that’s where the issue is. Does that make sense?
Thanks
–John
Hi @hokiegeek2
this is my configuration
extraEnv: |
- name: KC_SPI_X509CERT_LOOKUP_PROVIDER
value: nginx
- name: KC_SPI_X509CERT_LOOKUP_NGINX_SSL_CLIENT_CERT
value: "ssl-client-cert"
- name: KEYCLOAK_X509CERT_LOOKUP_NGINX_SSL_CERT_CHAIN_PREFIX
value: "USELESS"
- name: KEYCLOAK_X509CERT_LOOKUP_NGINX_SSL_CERTIFICATE_CHAIN_LENGTH
value: "2"
- name: KC_HTTPS_CERTIFICATE_FILE
value: "/etc/x509/https/tls.crt"
- name: KC_HTTPS_CERTIFICATE_KEY_FILE
value: "/etc/x509/https/tls.key"
ingress:
enabled: true
ingressClassName: nginx
servicePort: https
rules:
- # Ingress host
host: "keycloak-http"
# Paths for the host
paths:
- path: /
pathType: Prefix
annotations:
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/proxy-ssl-secret: "default/x509-secret"
nginx.ingress.kubernetes.io/proxy-ssl-verify: "off"
nginx.ingress.kubernetes.io/proxy-ssl-verify-depth: "2"
nginx.ingress.kubernetes.io/auth-tls-secret: "default/x509-secret"
nginx.ingress.kubernetes.io/auth-tls-verify-client: "optional"
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "2"
nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
hosts:
- keycloak-http
tls:
- hosts:
- "keycloak-http"
secretName: x509-secret
extraVolumes: |
- name: x509
secret:
secretName : x509-secret
extraVolumeMounts: |
- name: x509
mountPath: /etc/x509/https
I was able to do direct-grant with certificate
@mikhust unfortunately, same issue, being prompted for username/password, with the same error message that the header is empty:
Again, they truststore is loaded correctly, the x509 authenticator is registered correctly, but, if I am interpreting the error message correctly, the cert never makes it to keycloak.
Question: do you think this is my keycloak config still or do you think it’s an issue w/ the nginx ingress controller config?
@mikhust what are all the reasons that keycloak is reporting the HTTP header is empty? Keycloak is looking for an entry named “ssl-client-cert”, correct?
Hi @hokiegeek2 - have you created the secret with ca cert in it?
I am doing like this
#!/bin/bash
export PASSPHRASE=123456
CERTS_DIR="certs"
mkdir -p $CERTS_DIR
if kubectl get secret x509-secret; then
echo "X509 secret already exists"
else
#Generate CA and Server Certificates
openssl genrsa -aes256 -out $CERTS_DIR/rootca_kc.key -passout env:PASSPHRASE 2048
openssl req -x509 -new -nodes -key $CERTS_DIR/rootca_kc.key -sha256 -days 1024 -out $CERTS_DIR/rootca_kc.crt -subj "/C=US/ST=Home/L=Home/O=mycorp/OU=myorg/CN=caroot.keycloak-http" -passin env:PASSPHRASE
openssl genrsa -out $CERTS_DIR/tls.key 2048
openssl req -new -key $CERTS_DIR/tls.key -out $CERTS_DIR/keycloak-http.csr -subj "/C=UA/ST=Home/L=Home/O=mycorp/OU=myorg/CN=keycloak-http"
openssl x509 -req -extfile <(printf "subjectAltName=DNS:keycloak-http") -in $CERTS_DIR/keycloak-http.csr -CA $CERTS_DIR/rootca_kc.crt -CAkey $CERTS_DIR/rootca_kc.key -CAcreateserial -out $CERTS_DIR/tls.crt -days 500 -sha256 -passin env:PASSPHRASE
#Generate Client certificate
openssl genrsa -out $CERTS_DIR/john.doe.key 2048
openssl req -new -sha256 -key $CERTS_DIR/john.doe.key -out $CERTS_DIR/john.doe.req -subj "/CN=John"
openssl x509 -req -sha256 -in $CERTS_DIR/john.doe.req -CA $CERTS_DIR/rootca_kc.crt -CAkey $CERTS_DIR/rootca_kc.key -extensions client -days 365 -outform PEM -out $CERTS_DIR/john.doe.cer -passin env:PASSPHRASE
# Create Kubernetes Secret for Keycloak
kubectl create secret generic x509-secret --from-file=ca.crt=$CERTS_DIR/rootca_kc.crt --from-file=tls.crt=$CERTS_DIR/tls.crt --from-file=tls.key=$CERTS_DIR/tls.key
fi
Run octant to see if everything is in place
In my case empty header was due to missing ca cert
Good luck