If you are using the React SDK (which uses Hosted Backend: https://fusionauth.io/docs/apis/hosted-backend, then there are a couple options but they will all require some integration work from your end. The SDKs and Hosted Backend are designed to be easy to use and implement but they are not flexible as you can see with the cookies. Also I'm not sure if this was a consideration in the decision that running FusionAuth locally is not an option but just in case it was: You can use your FusionAuth non-production licenses wherever you want, we do not charge more "per deployment". So you can activate your non-prod license on a locally hosted FusionAuth instance in addition to your FusionAuth on Azure App Service, you can run your non-prod license on as many instances as you want.
Develop your application while hosting it on Azure App service so FusionAuth and the app are on the same domain
Setup a proxy for either your application or FusionAuth so they can be on the same domain
Documentation for setting up a proxy for FusionAuth:
https://fusionauth.io/docs/operate/deploy/proxy-setup
Create your own Hosted Backend, example here: https://github.com/FusionAuth/fusionauth-javascript-sdk-express/tree/main
Similar to #3, instead of setting up a Hosted Backend use the OAuth2 endpoints directly. In this scenario you will also be responsible for doing the OAuth code exchange for a token then setting the token cookies on the browser as well as session management with these tokens.
https://fusionauth.io/docs/lifecycle/authenticate-users/oauth/endpoints
https://fusionauth.io/docs/operate/secure/token-storage