Problems listing users synced from LDAP

Hi all,

we have a 3-node Keycloak cluster here with its main realm getting its users from LDAP.

The LDAP user federation is set to do a full, read-only sync every 12 hours and has a cache policy of DEFAULT (I played with various other cache settings, but to no avail). The imported LDAP user base is ~600 users in ~800 hierarchical groups.

The setup works well and is reasonably fast for authentication purposes and to query individual user details.

My problem is that I need to list users via the REST API (using the Java Keycloak library) and I’m running into problems there:

1. Speed

Retrieving a list of all 600 users takes in excess of 10 minutes. Trying to do it without paging (by setting the max sufficiently high) gives very inconsistent results (randomly stopping somewhere after between 250 and 350, without actually giving me an error). With paging, it’s just still terribly slow.

I can’t figure out what’s so slow here, whether it’s Keycloak itself, or whether it tries to refresh the users from the LDAP while listing for some arcane reason, or something else.

Any pointers to diagnose and speed this up would be greatly appreciated. Ideally, I’d like Keycloak to answer all of this from its local user base without reaching out to LDAP (outside its scheduled syncs), and do it fast of course.

2. Group membership

Since I actually don’t need all the 600+ users, but just those in a specific group (maybe 100), I thought I could simply list the group members instead, hoping this would be faster and all-around more efficient.

Unfortunately, this fails with an exception:

ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-277) Uncaught server error: com.fasterxml.jackson.databind.JsonMappingException: [no message for java.lang.NullPointerException]
	at com.fasterxml.jackson.core.jackson-databind@2.12.1//com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
	at com.fasterxml.jackson.core.jackson-databind@2.12.1//com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
	at com.fasterxml.jackson.core.jackson-databind@2.12.1//com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
	at com.fasterxml.jackson.core.jackson-databind@2.12.1//com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1510)
	at com.fasterxml.jackson.core.jackson-databind@2.12.1//com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1006)
	...
Caused by: java.lang.NullPointerException
	at org.keycloak.keycloak-model-infinispan@13.0.1//org.keycloak.models.cache.infinispan.UserCacheSession.getUserByUsername(UserCacheSession.java:243)
	at org.keycloak.keycloak-ldap-federation@13.0.1//org.keycloak.storage.ldap.LDAPStorageProvider.loadUsersByUsernames(LDAPStorageProvider.java:418)
	at org.keycloak.keycloak-ldap-federation@13.0.1//org.keycloak.storage.ldap.mappers.membership.MembershipType$1.getGroupMembers(MembershipType.java:119)
	at org.keycloak.keycloak-ldap-federation@13.0.1//org.keycloak.storage.ldap.mappers.membership.group.GroupLDAPStorageMapper.getGroupMembers(GroupLDAPStorageMapper.java:576)
	at org.keycloak.keycloak-ldap-federation@13.0.1//org.keycloak.storage.ldap.LDAPStorageProvider.lambda$getGroupMembersStream$4(LDAPStorageProvider.java:396)
	...
	at org.keycloak.keycloak-ldap-federation@13.0.1//org.keycloak.storage.ldap.LDAPStorageProvider.getGroupMembersStream(LDAPStorageProvider.java:399)
	at org.keycloak.keycloak-server-spi@13.0.1//org.keycloak.storage.user.UserQueryProvider$Streams.getGroupMembersStream(UserQueryProvider.java:655)
	at org.keycloak.keycloak-services@13.0.1//org.keycloak.storage.UserStorageManager.lambda$getGroupMembersStream$8(UserStorageManager.java:265)

From the code, this seems to happen because the username is null. I checked the Keycloak database and there is no user_entity with an empty or null username (increasing my suspicion that Keycloak sneaks out calls to LDAP here and there’s a problem in LDAP or in the query somehow).

Any pointers to get to the bottom of this NPE and possibly fix any problematic LDAP data?