You don’t need insecure http access, when you have reverse proxy with TLS offloading. Just access keycloak via proxy (of course Keycloak must be aware that is behind reverse proxy, e.g. PROXY_ADDRESS_FORWARDING=true). It is also good idea to have encrypted connection also after reverse proxy.
Good starting point for AWS ECS (Fargate) is: