Troubleshooting

Overview

If any problems arise or if you are unable to access FusionAuth, consult the FusionAuth log files. In most cases API errors will be written out to the fusionauth-app.log file, but for some installations such as Docker, the errors will be sent to stdout.

If you need further assistance, please ask a question in the FusionAuth forum or open an issue on Github if you have found a bug (please provide replication steps).

If you have a paid edition which includes support you may open a support request from your FusionAuth Account page. Learn more about purchasing support.

Here’s more information about getting technical support.

Logs

System Log UI

The system logs may be reviewed in the FusionAuth admin UI by navigating to System → Logs. This feature was added in version 1.16.0. This UI will only render logs for systems that write their logs out to disk. Deployments which write logs to STDOUT, such is the case with Docker, will not be able to review logs through this interface.

The system log UI organizes and renders the most recent 64KB of all logs for all nodes in the deployment. If you need the entire log, use the download action in the upper-right hand corner of UI to download a zip of all logs.

System Logs - Single Node System Logs - Multiple Nodes

Filesystem Logs

Alternatively, the logs may be accessed directly. The following are the default locations for each of the FusionAuth log files. You may or may not have all of the FusionAuth services installed for your system, so you may not have all of the following logs on your server.

Linux and macOS
/usr/local/fusionauth/logs/fusionauth-app.log
/usr/local/fusionauth/logs/fusionauth-search.log

These paths assume the suggested product location of \fusionauth. This path may be different on your system depending on where you unpacked the zip files.

Windows
\fusionauth\logs\fusionauth-app.log
\fusionauth\logs\fusionauth-search.log

Note that if you started Windows via Fast Path, the fusionauth-app.log file will not be created. Instead the services are running interactively and all logging is written to to stdout.

Event Log

Available since 1.6.0

The event log is a FusionAuth message store designed to capture different types of events that may be necessary to communicate to a FusionAuth developer or admin user.

The event log may contain helpful details to indicate the cause of the failure, or a failure condition you need to be aware of in FusionAuth. See System → Event Log.

While not limited to, generally speaking the event log will contain events or errors related to external systems or asynchronous issues that are difficult to communicate to the API caller or the FusionAuth admin at runtime. While not intended to be an exhaustive list, examples of these types of errors are:

  • SMTP connection issues

  • Lambda invocation errors

  • External identity provider failures or configuration issues

  • Failure to deliver a webhook event

Troubleshooting Email

FusionAuth sends a lot of email, for forgotten passwords, passwordless login and other features.

Troubleshooting email delivery is difficult. There are many factors affecting it. However, there are steps you can take to narrow down the problem.

Send A Test Email

The Send Test Email button has been available since 1.16.0

The first step is to ensure that you can send a test email. Navigate to Tenants → Your Tenant → Edit → Email and send a test email. If it is received, FusionAuth can send emails via SMTP.

Sending a test email

If sending a test email fails, a tool such as SWAKS can help debug SMTP issues. Using this tool removes FusionAuth from the equation.

Further Investigation

If the test email succeeds, but you aren’t receiving other emails, investigate further by following these steps:

  • Send emails to different addresses at different providers and check the spam folder.

  • Make sure that you are firing the event that you expect to send email correctly. For instance, if you are looking for email verification of users, make sure that is enabled.

  • Test with a different FusionAuth function. Sending a password reset email is easy to do from the user details screen.

  • Make sure the email template is correct, or use a default template.

SMTP Logging

If you would like to see verbose SMTP logging, follow these steps:

  1. Enable debugging by navigating to Tenants → Your Tenant → Edit → Advanced → SMTP Settings → Additional properties and add mail.debug=true.

  2. Save the tenant.

  3. Send an email.

  4. View the system logs by navigating to System → Logs.

  5. Select fusionauth-app.log and you will see verbose SMTP output.

Doing this logs the full SMTP conversation, which can be verbose. You should remove this setting when you have finished troubleshooting.

Troubleshooting SAML or OIDC Identity Provider Configuration

A common pattern is to set up SAML or OIDC as the login method for the FusionAuth application. This lets you keep all your users in the SAML or OIDC system, and have FusionAuth delegate auth decisions to it. However, if your Identity Provider isn’t configured correctly, you can end up in a dead end, unable to log in. This might happen if you:

  • Set up SAML or OIDC as the authentication method for your FusionAuth admin application.

  • Set the Managed domains to your domain. For example, example.com.

  • Don’t have any users with a domain that is different than example.com.

  • Accidentally misconfigure SAML or OIDC and break authentication.

  • Log out of the FusionAuth admin application.

You won’t be able to log in to the FusionAuth admin application to correct the misconfiguration. You’ve locked down the FusionAuth admin application so that only valid logins can access it, but because of the misconfiguration, there is no account with a valid SAML or OIDC login.

Unable to login with a managed domain and misconfigured Identity Provider.

This is a problem, but not an insurmountable one. Your options depend on when you discover the issue. If you are beginning your SAML or OIDC configuration, you can avoid this scenario. Follow these steps:

  • Don’t set any value of Managed domains before you have tested the SAML or OIDC configuration.

  • Test authentication in a different or incognito browser window, ensuring that an admin user account is always accessible to modify configuration.

  • Add an admin user account with a domain not in the Managed domains setting. Ensure this user is registered with the FusionAuth admin application and has the admin role.

  • Set up an API key. Navigate to Settings → API Keys to do so and make sure it has privileges to create users and registrations. This will open up options in the future to reset settings without using the administrative user interface.

If you are currently locked out of your FusionAuth application, you have fewer options:

  • If you have a known username and password that are not delegated to SAML or OIDC (perhaps the initial account created when you set up FusionAuth) proceed to the FusionAuth admin login page. Append &showPasswordField=true to the end of the login URL. This will force the UI to show the password field.

  • If you have an API key with appropriate privileges, you can modify the configuration without using the administrative user interface. Add an admin user with a different email domain and sign into the admin interface to correct your SAML or OIDC configuration. Here’s how you might do so (3c219e58-ed0e-4b18-ad48-f4f92793ae32 is the FusionAuth application Id):

Adding a New Admin User With a Different Domain
curl -XPOST -H "Authorization: $API_KEY" -H "Content-Type: application/json" \
-d '{"user": {"email": "user@differentdomain.com","password":"password"}, "registration": {"applicationId" : "3c219e58-ed0e-4b18-ad48-f4f92793ae32","roles":["admin"]}}' \
https://local.fusionauth.io/api/user/registration

If you have neither an API key nor a known user, you can restore from a database backup, modify the database directly if you have access, or re-install FusionAuth.

Troubleshooting Themes

If you happen to get into a situation where you have edited a template and it is causing errors that are preventing you from logging in, you can override the use of the UI templates to render a login form that lets you log in.

To do this, open your browser and access your FusionAuth admin UI. This will redirect you to the broken /oauth2/authorize page. Click in your browsers address bar and scroll to the end. Finally, add the String &bypassTheme=true to the end of the URL and hit the Enter key. This should render the default login page that ships with FusionAuth and allow you to log in and fix any errors you have.

Troubleshooting Two Factor Authentication

If you are receiving an invalid code for your two factor authentication and you are using a time based one time password (TOTP) application such as Google Authenticator or Authy, confirm that the system time is correct on the server.

The TOTP two factor auth system is time dependent and if there is any slippage between the system time of the client and the system time of the server, the generated code will not be correct. This is also known as "clock skew".

The fix is to ensure that the FusionAuth server has the correct system time. How exactly do that that depends on the type of system; please consult your operating system documentation.

Troubleshooting Database Connections

If you find that FusionAuth is unable to connect to the database, ensure you are able to use a command line client connection to successfully make a connection using the same port and credentials.

Some MySQL services such as Microsoft Azure may require a specific version of TLS to connect successfully. At the time of writing this note, the MySQL connector will not attempt to utilize TLSv1.2 by default, so when connecting to a service that requires this version you will need to explicitly request this version of TLS on the connection string. For example, appending this enabledTLSProtocols=TLSv1.2 to the connection string will allow you to successfully connect to an Azure managed MySQL database. See MySQL Connector : Connecting Securely Using SSL for additional information.

Common Errors

Access Denied

If you see an Access Denied error in your browser while using the FusionAuth UI, this could be caused by a mis-configured CDN or proxy. This error often produces a 4xx error code such as a 403. In order to fix this issue, ensure that your CDN, proxy or gateway does not prevent traffic from flowing directly to FusionAuth. FusionAuth is able to handle all HTTP traffic and any network handling between the browser and FusionAuth should be as simple as possible.

FusionAuth Stops Unexpectedly

If you are running FusionAuth on a server with limited memory, it may stop unexpectedly. This can also happen when there are many other applications competing for memory on the server.

When FusionAuth is killed because of an out of memory issue (OOM), it dies with no explanation in the FusionAuth logs. You might see lines like this in server system logs (typically under /var/log/): 

Dec 30 12:00:38 vps kernel: Out of memory: Kill process 30047 (java) score 98 or sacrifice child

The OOM killer will begin killing services once the kernel runs out of memory. The solution will be to allocate less memory to FusionAuth or to increase the amount of RAM available.

You can do the former with the fusionauth-app.memory setting. See the configuration reference for more details.

You may do the latter by running a larger server or running fewer competing applications. In particular, applications used by FusionAuth such as Elasticsearch or the database may be moved off to external servers.

Feedback

How helpful was this page?

See a problem?

File an issue in our docs repo