Multiple IDPs using RSA signed JWTs in one realm


We’re using keycloak as our internal OIDC-server, and we’re federating two different IDPs that both use JWT signed with private RSA key. This key is as I’ve understood set on a realm level, and not configurable per IDP. Does this mean that it’s impossible to integrate with these two IDPs without using two realms?