Disable cache for HA mode

We would like to setup a highly scalable HA mode for Keycloak with a pretty simple scenario:
User sign-up/in with username/password, etc…

Due to the expected high load on the app we’ve configured Keycloak to run in HA mode with external Postgres DB in AWS. However, we would like Keycloak to run only in stateless mode and in order to scale horizontally we would like to depend on db rather than the Keycloak as app.

My question is - is the above a good strategy (given the fact that we will sacrifice performance a bit)? For the purpose we would like to disable all caches (if possible) and to not apply any configuration in that regards.
Is it a good idea for production setup, shall we expect some issues with node discovery in a cluster if all cache is disabled?