Hiya,
This looks like a similar question: https://fusionauth.io/community/forum/topic/360/example-docs-for-fusionauth-integration-as-sso-front-end-to-nginx-proxied-apps
It looks like authelia is slightly different and has tighter integration with nginx, but no support for OIDC.
It looks like nginx works with OIDC, but only with NGINX plus (which I believe is the version that costs money?): https://github.com/nginxinc/nginx-openid-connect
If you want to integrate with OpenLDAP, you'd be looking buying a premium version of FusionAuth (Developer or above) because the community edition doesn't support LDAP integration. See the Connectors documentation for more.
We have an example plugin here for apache, I'm not super familiar with nginx, but maybe you could do something similar: https://github.com/FusionAuth/fusionauth-mod-authnz-external
Another alternative would be to see if the apps you have have OIDC/SAML integration and directly connect them with FusionAuth, though that doesn't provide the URL level endpoint protection you asked for (only application level).