Error 403 with Nginx in front

zradick

@dan Thanks, we are running the nginx tightly coupled with our fusion auth deploy (we modified the Docker file in order to start both together in a single container) so that we can deploy it to AWS Fargate. I'm not sure exactly why we were getting a 403 hitting the admin endpoint for auth through the LB, however we are closing off ingress to the public internet anyway and we are able to successfully auth when tunneled through our bastion host and the application API is working so we can at least upgrade to 1.41.3 for the security patch there.

johnmiller

Hi @dan,

Is there any update on this issue?

BR,
John

dan

@johnmiller Hiya,

Did you try the fix mentioned here: https://fusionauth.io/community/forum/topic/2254/having-an-issue-with-nginx-in-front-of-fusionauth

If not, what issue are you seeing in particular?

rickberon

The Nginx 403 Forbidden error occurs when a client attempts to access a section of the webserver without adequate permissions. When Nginx accesses a directory, it attempts to generate an index of its files and present them to the browser or client. However, the default configuration disables directory indexing, leading to the display of the 403 forbidden error instead.

Incorrect Index File

The try_files tries the literal path you specify in relation to the defined root directive and sets the internal file pointer. If you have directory indexing off, and is having this problem, it's probably because the try_files you are using has a directory option:

location / {
  try_files $uri $uri/ /index.html index.php;
}

to

location / {
  try_files $uri /index.html index.php;
}

Incorrectly set permissions

This error can also result from files and directories having incorrectly set permissions. In order to resolve this , change the directories permission to 755 and the file permissions to 644 . Make sure that the user running the Nginx process owns the files. For example, set user to www-data:

sudo chown -R www-data:www-data *

Finally, set the directory and file permissions as:

sudo chmod 755 {dir}
sudo chmod 644 {files}

mark

I'm having the same problem. Unable to solve it so far.
@johnmiller @maburns @zradick did you manage to get past this problem, and if so, how?
Thanks for any advice!

mark

Here's some info about our setup.

FA behind nginx reverse proxy, docker based. 3 environments (dev, staging, production). Has been working for many months without trouble.
A few days ago, upgraded FA in the DEV env, from 1.28 to 1.47.1
Since then, when trying to log into FA, the POST request to /oauth2/authorize is responded by a blank page (HTTP 403)
Tried a lot of stuff, but to no avail so far. We're using the recommended nginx config (https://github.com/FusionAuth/fusionauth-contrib/blob/master/Reverse Proxy Configurations/nginx/nginx.ssl.conf)
For debugging purposes, I've built up a 4th environment with the exact same setup (nginx reverse proxy and FA). The FA installation (1.47.1) was installed from scratch. The problem is not showing in that environment.
This leads to the suspicion that the 403 could be related to existing data/config in the existing installation. But which? And how to access logs, when not being able to log into FA?
The logs in stdout of the FA container are not anything related to the error.

Any ideas? How can I see logs of why FA decides for a 403 in /oauth2/authorize ?

mark.robustelli

@mark Are you able to access the system through the APIs? https://fusionauth.io/docs/v1/tech/apis/system#export-system-logs

mark

@mark-robustelli Unfortunately, no. We're running FusionAuth in a Docker container. For that scenario, the doc says "When running FusionAuth on Docker or other container service where logs are written to stdout and not written to the file system, this API will return an empty archive.".

mark.robustelli

@mark I know this is a long shot, but can you access other pages after you attempt to log in as is occuring with this post https://fusionauth.io/community/forum/topic/2488/dashboard-homepage-timeout/3 ?

mark

@mark-robustelli Unfortunately not. I've tried it several times, with different browsers (to make sure it's not a cache issue etc.). It seems that when logging in, the page https://<myurl>/oauth2/authorize (which results in a 403) is not really logging me in. If afterwards I go to https://<myurl>/admin/application or https://<myurl>/admin/tenant, I'm still redirected to the login form.

I really would like to see FusionAuth's application logs, I'm pretty sure I'd see something there why the 403 occurs. Is there really no way? I need to get this issue fixed.

dan

You can set up a Docker log driver to direct log output somewhere else:

mark

OK so I was finally able to track it down fix it. Some short notes below, maybe it helps anyone else running into the same problem.
Thanks to @mark-robustelli and @dan for their inputs.

I accessed FusionAuth directly on port 9011, completely bypassing the reverse proxy
In the CORS filter config, I activated "Debug enabled" which in the system logs showed me java.lang.IllegalArgumentException: Illegal character in authority at index 7 and java.lang.NullPointerException: Cannot read field "actionURI" because "actionInvocation" is null
Going from there, I completely disabled the CORS filter
This immediately got rid of the 403 and everything worked again, accessing FA using the reverse proxy
The FA UI told me that I'm missing two headers in my reverse proxy config: X-Forwarded-Proto and X-Forwarded-Port, which I then configured (https/443 in my case)
After that I enabled the CORS filter again (all default values) and since, everything is working correctly, like before

dan

@mark Awesome!