Client usage restricted by group

I’m trying to restrict access to a client based on group membership of a user. I configured it as specified here:

I setup the group, made sure I was not a member of the group, setup a website using oauth2_proxy. It lets me login and access the site even though I’m not a member of the group. What am I missing?