Users Are unable to login after we moved from ADFS to Azure AD

Note: Ignore # , these are used only to treat more than 2 links error for this keycloak forum.

Previous setup -
Previously we were having our Keycloak 18.0.2 trust federated with On-prem ADFS and users were managed in on prem Active directory. Users were logging in using their adfs identity like, domain\username. So these users were stored in keycloak. Their email was like #username@domain.com#

Changes we made -
Now we setup our azure AD in cloud. We also synced our on prem AD with Azure AD. Means azure ad domain is same as of on prem AD like both are example.com .

Problem we are facing -
Now those users are facing issues like User account already exists and we are getting error logs of IENTITY_PROVIDER_FIRST_BROKER_LOGIN_ERROR. We observe that users are using their old format usernames like domain\username.

What we did to resolve it ? -
We deleted those users who were facing above mentioned error. Then they were able to log in.

Our expectation-
keycloak should still grant them access without deleting previous entry as we have synched AAD and ADFS. Both the places their username in format #username@domain.com# is same.
And in case of adfs whether you use domain\username or username@domain.com , it still consider it as same user.

So, what setting we should change in keycloak so we can achieve it ?