LDAP Filter to restrict token generation

We are developing a service that will only be accessible to a subset of our larger organization. We are using Keycloak to secure the service, but we are using the organization’s LDAP for users. We thought that by putting in the Custom User LDAP Filter in the User Federation Keycloak would only authenticate users in the filter. We are seeing that anyone in the organization’s LDAP is getting a token, and adding the user to the local Keycloak user database. Is the filter functioning as expected, and the clients need to be restricted to have Keycloak check for authorization (that we would then set up through group mapping)?