Managing large amount of resources

Hi,

We manage documents. Customers are going to create thousands (100k per year) of documents which are confidential. So by default access should be denied and the owner can determine who has access to his/her document. The fun thing is that this not on a group level but on a personal level.

That brings up two questions:

  • Do we need to create a resource per document or can this be handled smarter?
  • Does that also mean you get a policy per user which is assigned to the resource via a permission? Wouldn’t this mean that if we have 10k accounts, that we have 10k policies?

The dots don’t seem to connect for me and some help would be great :slight_smile:

Thanks you!

Hello @waltertamboer ! Have you found an answer for this ? I’m on the same boat, where I have to create millions of resources in keycloak

Hi @espork . Unfortunately I did not find a solution yet.

At this point we use Keycloak for authentication only since it’s not clear how to handle that amount of resources we want. I think this topic is related: Checking permissions

Basically I don’t mind creating this amount of resources and user policies since it would be done automatically. But it would be nice to get some validation from people that have done this already. The docs don’t seem to mention any good examples either.

The topic mention earlier is still open. If we have a good answer for that, than this question will answer itself I guess.

Maybe we can share ideas to come up with a solution together?

Hello @waltertamboer thanks for the reply . let me give you a background about what we are trying to do : I’m using Keycloak with a gateway in front of an service that can have millions of resources .
The idea is to have a resource be created in keycloak and in the service in order to have Keycloak controlling access to those resources (URIs).
For example, a user A creates 1 million RESOURCE_X , so I’d have 1MI resources in keycloak with uri and USER permission ( only owners can access the resources for now ) and also one million of RESOURCE_X in my backend.
My question is : Is there a limit of resources with URI the keycloak can handle and be fast during permission check of the uri of the resource ? I did some load test and specially when I’m creating a new resource, keycloak verifies permission and it takes quite a lot to figure out that the URI does not existe yet in the resources collections and the grants access ( as Default resource grants access to any URI that is no owned by anyone - not created yet)

Right now the problem we see is that when the uri (resource) is not in keycloak yet, that means we will create it in the resources and also in our back end, keycloak takes a lot of time to check permission on something that is not in the list yet . We have added more than 10k resources so far and it adds a big overhead on resource creation for our back end as the number of resources increases in keycloak.

Hi @waltertamboer, @espork, I have exact same problem and after several days of googling, I couldn’t find a single clarification about this, in my eyes quite common and straightforward, problem. Has any of you manage to find some more info about it?