Logo / Docs

Deploy FusionAuth to a Kubernetes Cluster

Kubernetes is an extensible and open-source container-orchestration system used for automating application deployments. Some of the most notable features it provides include self-healing, auto-scaling, and automated rollouts and rollbacks.

Since the Kubernetes container runtime supports Docker, FusionAuth Docker containers can be deployed to any Kubernetes cluster.

The following guide demonstrates how to deploy FusionAuth containers using kubectl, the Kubernetes command-line tool that enables you to send commands to a cluster through the API server. You’ll need a cluster running to work through this guide.

Requirements

Adding the Helm Chart Repository

To get started, the first thing to do is add the FusionAuth Helm Chart repository. This can be done with the following.

Add a chart repository

helm repo add fusionauth https://fusionauth.github.io/charts

To view the list of repositories that you have added use the following command:

List chart repositories

helm repo list

Output from listing chart repositories

NAME      	URL
fusionauth	https://fusionauth.github.io/charts

To search repositories:

Search chart repositories

helm search repo fusionauth

Output from search

NAME                 	CHART VERSION	APP VERSION	DESCRIPTION
fusionauth/fusionauth	0.10.5       	1.30.1     	Helm chart for fusionauth

Helm Chart Configuration

Before you install, configure the values.yaml file contents used by the Helm Chart. The majority of the values for this chart have defaults recommended by FusionAuth but you will want to review and modify the configuration to meet your specific requirements.

To update this file, first download it:

Download chart values

curl -o values.yaml https://raw.githubusercontent.com/FusionAuth/charts/main/chart/values.yaml

Open values.yaml with your favorite text editor and modify it. This guide covers configuration values which must be updated to deploy FusionAuth. You should review the entire configuration to understand all the possible options.

Replicas

The replicaCount value indicates the number of instances of FusionAuth to run in a ReplicaSet. Kubernetes will monitor the cluster to ensure that this number of instances of FusionAuth is running at all times.

By default, the replicaCount value is set to 1 which typically works well for development and testing environments, but you will likely want to adjust this based on your system and application requirements.

Replica count example configuration

# replicaCount -- The number of fusionauth-app instances to run
replicaCount: 1

Docker Image

The most current helm chart will always point to the latest version of FusionAuth. For this guide, FusionAuth version 1.30.1 is used. You can update this to any released version you’d like. You can find the list of FusionAuth releases here.

Image configuration example

image:
  # image.repository -- The name of the docker repository for fusionauth-app
  repository: fusionauth/fusionauth-app
  # image.repository -- The docker tag to pull for fusionauth-app
  tag: latest

Extra Containers

You may want to inject a sidecar or ambassador container. This is useful if you want to, for instance, proxy requests to FusionAuth to put it at a different path or limit the IP addresses which have access to the administrative user interface. You can do this with the extraContainers value. Containers defined here will be added to the FusionAuth pod.

Default extraContainers

extraContainers: []

extraContainers with an nginx sidecar

extraContainers:
  - name: my-sidecar
    image: nginx:latest

You can find community contributed proxy configurations here.

Database Configuration

Now, configure the database connection. Each of our FusionAuth pods will use this configuration when communicating with the database.

The following is an example of the database configuration section in values.yaml:

Database and search example configuration

database:
  # database.protocol -- Should either be postgresql or mysql. Protocol for jdbc connection to database
  protocol: postgresql
  # database.host -- Hostname or ip of the database instance
  host: ""
  # database.host -- Port of the database instance
  port: 5432
  # database.tls -- Configures whether or not to use tls when connecting to the database
  tls: false
  # database.tlsMode -- If tls is enabled, this configures the mode
  tlsMode: require
  # database.name -- Name of the fusionauth database
  name: fusionauth

  # To use an existing secret, set `existingSecret` to the name of the secret. We expect at most two keys: `password` is required. `rootpassword` is only required if `database.root.user` is set.
  # database.existingSecret -- The name of an existing secret that contains the database passwords
  existingSecret: ""
  # database.user -- Database username for fusionauth to use in normal operation
  user: ""
  # database.password -- Database password for fusionauth to use in normal operation - not required if database.existingSecret is configured
  password: ""
  # These credentials are used for bootstrapping the database and creating it if needed. This can be useful for ephemeral clusters used for testing and dev.
  root:
    # database.root.user -- Database username for fusionauth to use during initial bootstrap - not required if you have manually bootstrapped your database
    user: ""
    # database.root.password -- Database password for fusionauth to use during initial bootstrap - not required if database.existingSecret is configured
    password: ""

search:
  # search.engine -- Defines backend for fusionauth search capabilities. Valid values for engine are 'elasticsearch' or 'database'.
  engine: elasticsearch
  # search.engine -- Protocol to use when connecting to elasticsearch. Ignored when search.engine is NOT elasticsearch
  protocol: http
  # search.host -- Hostname or ip to use when connecting to elasticsearch. Ignored when search.engine is NOT elasticsearch
  host: ""
  # search.port -- Port to use when connecting to elasticsearch. Ignored when search.engine is NOT elasticsearch
  port: 9200
  # search.user -- Username to use with basic auth when connecting to elasticsearch. Ignored when search.engine is NOT elasticsearch
  user: ""
  # search.password -- Password to use with basic auth when connecting to elasticsearch. Ignored when search.engine is NOT elasticsearch
  password: ""

For this guide, the following attributes need to be retrieved from your database and Elasticsearch deployments and updated in your values.yaml configuration.

FusionAuth requires an external database and, optionally an external Elasticsearch cluster. Setting these up is outside of the scope of this document.

Consult the setup guides for instructions for AWS, GCP, and more.

  • database.host
  • database.root.user
  • database.root.password

The FusionAuth standard user will be created with the credentials provided here when you install the FusionAuth helm chart.

  • database.user
  • database.password

If search.engine is set to elasticsearch:

  • search.host

Additionally, you may set these values if a username/password are needed to access Elasticsearch (these are not required for this tutorial):

  • search.user
  • search.password

If you don’t want to provide the database.root.user and database.root.password to allow FusionAuth to set up the database itself, you can also follow the advanced database installation guide which lets you set up the database out of band.

Deploy FusionAuth To the Cluster

Now that values.yaml is updated, it is time to install the chart on the cluster. The helm install command is used to install a chart by name and can be applied using the syntax:

Helm install format

helm install [CHART NAME] [CHART] [flags]

Here you will install a chart including the -f flag to override the default values.

Install the FusionAuth chart

helm install my-release fusionauth/fusionauth -f values.yaml

You can apply override values by modifying the values.yaml configuration that you pass when installing a chart or by using the --set flag on the command line. Using --set can be useful when developing scripts or command line tools. Either method is acceptable.

If the previous command was successful, you should see output similar to the following:

Example output

NAME: my-release
LAST DEPLOYED: Sun Oct 10 19:23:41 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export SVC_NAME=$(kubectl get svc --namespace default -l "app.kubernetes.io/name=fusionauth,app.kubernetes.io/instance=my-release" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:9011 to use your application"
  kubectl port-forward svc/$SVC_NAME 9011:9011

You can get the status of your deployment with kubectl.

Get a list of deployments running on the cluster

kubectl get deployments -o wide

Output

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                             SELECTOR
my-release-fusionauth   1/1     1            1           4m16s   fusionauth   fusionauth/fusionauth-app:1.30.1   app.kubernetes.io/instance=my-release,app.kubernetes.io/name=fusionauth

As instructed by the success message output above, you can create a proxy enabling you to connect to the cluster from localhost:

Setup port-forwarding proxy

export SVC_NAME=$(kubectl get svc --namespace default -l "app.kubernetes.io/name=fusionauth,app.kubernetes.io/instance=my-release" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:9011 to use your application"
  kubectl port-forward svc/$SVC_NAME 9011:9011

Example output of kubectl proxying command

Forwarding from 127.0.0.1:9011 -> 9011

Navigate to http://localhost:9011 and you will land on the FusionAuth Setup Wizard. For a complete tutorial walking you through the initial FusionAuth configuration, see Setup Wizard & First Login.

FusionAuth bootstrap

Create an Ingress

If you want to connect FusionAuth with an external network, the last thing you need to do is to configure the cluster to be able to receive external requests. You might do this if you were using FusionAuth to authenticate users connecting to applications from the internet. On the other hand, you might not do this if you were using FusionAuth solely within your kubernetes cluster.

To direct external traffic to your cluster, you will use an Ingress, a component that defines how external traffic should be handled, and an Ingress Controller that implements those rules.

Deploy an Ingress Resource

Create a file named ingress.yaml and copy and paste the following resource configuration:

Ingress resource configuration

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-release-fusionauth
  labels:
    app.kubernetes.io/name: fusionauth
    app.kubernetes.io/instance: my-release-fusionauth
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: my-release-fusionauth
            port:
              number: 9011
        path: /
        pathType: Prefix

The rules for this Ingress resource indicate that requests from the root path context, or /, should be directed to the my-release-fusionauth service.

Now apply this configuration on the cluster:

Apply Ingress resource

kubectl apply -f ./ingress.yaml

Install an Ingress Controller

For this guide, you will use the NGINX Ingress controller. To install the Ingress controller, add the nginx-ingress repository and install the Helm chart by running the following commands:

Add ingress-nginx chart repository

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

Install ingress-nginx chart

helm install fa-loadbalancer ingress-nginx/ingress-nginx

When completed, get the active services running on the cluster:

Get services information

kubectl get services

Services response output

NAME                                            TYPE           CLUSTER-IP       EXTERNAL-IP                                                                  PORT(S)                      AGE
fusionauth-ingress-nginx-controller             LoadBalancer   10.100.246.131   a4413292c6477456b8c1cde8f5260513-137482286.us-west-1.elb.amazonaws.com       80:30048/TCP,443:32484/TCP   37m

A new service will be available of type LoadBalancer and will contain an external IP or host name corresponding to a newly provisioned Elastic Load Balancer host name. External traffic will now be directed to the Kubernetes cluster via this ingress.

If you have completed this tutorial using minikube, you can access the ingress by creating a network route to your cluster by executing the command minikube tunnel. For documentation on this, see Using minikube tunnel.

Congratulations! You now have a fully functional Kubernetes environment running FusionAuth.

Troubleshooting

Runtime Mode Mismatch

If you receive a message similar to this one in your system log:

ERROR io.fusionauth.api.service.system.NodeService - Node [c98263dc-7f12-4879-93c1-f1a634912a51] cannot be added. The runtime mode is [Development] and this node in [Production] mode.

This means that your nodes are in different runtime modes. This will not work and may cause undetermined behavior. All nodes of a FusionAuth cluster must be running in the same mode, either Production or Development.

To resolve this issue, take the following steps:

  • Stop all existing nodes, whether they are running as containers or processes.
  • Review configuration to confirm that all nodes are configured with the same runtime mode.
  • Restart the nodes.

Next Steps

To learn more about Kubernetes, here are a few recommended links to help you administer your cluster.