Getting cors error when calling userinfo endpoint with expired access token

Hello. I am making a simple html and javascript page with different calls to keycloak. I can login, get tokens, refresh tokens and user info. My issue is that if my access token runs out, I am getting a cors error instead of a keyclaok invalid token error when I call user info. I would like to avoid the cors error somehow. It is keyclaok versoin 11

This is the code that is calling the userinfo endpoint

The * and localhost has been added to web origin