Context: We are in the process of migrating our legacy authentication system to the Keycloak (KC later in the text), and we are using our custom UserStorageProvider to provide users from our legacy database to the KC.
Note: We’re talking about a million users in one realm. There’s no possibility to use many realms at this moment.
We’ve come to a few problems while migrating our custom UserStorageProvider from the KC 11.0.3 to 12.0.2 version. Old query methods from the UserQueryProvider interface are marked as deprecated and advised to be replaced with the new methods containing the “Stream” suffix.
We’ve done as instructed and the first problem that we encounter was an “out of memory” error. Digging through the KC source code we think that we found the culprit.
Here it is: UserStorageManager_12.0.2.
The arguments firstResult and maxResults are no longer passed to the searchForUserStream method of our custom provider (lines 410 and 412), so our query has no limit anymore and fetches all the data from the DB causing the memory overflow.
This was not a problem on version 11.0.3: UserStorageManager_11.0.3.
We’ve managed to circumvent this by limiting our ResultSet to fetch the 1000 records per DB roundtrip, but now the fetching takes a lot longer than before (many roundtrips to the DB) and this slows down the listing of the users on the KC Admin app to almost 10s per page.
Any ideas of what could we do to improve this? Or we should stick to version 11.0.3?
Thanks for any help,