SAML Binding URL

I have two SAP systems which use different URLs but the same SAML provider (same Client Id). The problem is, that the response is sent to the stated Assertion Consumer Service POST Binding URL in the SAML client config. So SAML is only working for initiated login by domain1 or domain2 but not for both. When I change the domain in the SAML client config it is working for the related application instance. Do you have any idea, how to configure that the SAML response is sent to the domain requested the SAML login?