Custom authenticator provider driven by mobile app

We would like to do POC where we implement a custom authenticator provider for OTP.
All the examples and documentation I saw are all based on browser flow.
We have a mobile app and we don’ want to use browser flow, but instead using API calls from the mobile to drive the OTP authentication flow.
Any explanation high level of the API, existing examples or documents for such use case will be great.