Migration From Keycloak

Obtaining User Data

You will need read only database access to your Keycloak database. As part of the planning process, gather the credentials that will allow you to run SQL queries against this database.

You will need read only access to these tables:

• USER_ENTITY

• CREDENTIAL

You will also need to have a list of realms that you want to move and their names. You can find their names in the administrative user interface:

Make note of each realm name containing users you want to migrate. You may migrate one, many or all realms depending on your requirements and larger migration plan.

Mapping User Attributes

The attributes of the User object in FusionAuth are well documented.

If there is an attribute in your Keycloak user which cannot be directly mapped to a FusionAuth attribute, you can place it in the user.data field. This field can store arbitrary JSON values and will be indexed and searchable.

Social Logins

Keycloak also provides integrations with other social login providers such as Twitter, Google or Facebook. Review the supported FusionAuth Identity Providers to ensure your social providers are supported.

If not supported explicitly, a provider may work with an OIDC or SAML connection. Otherwise, please open a feature request.

Migrating users with social logins such as Apple or Facebook requires that you have an existing user Id for that provider. What this unique user Id looks like depends on the particular social identity provider. The unique Id may be an email address, an integer, UUID, or a random string.

Configure the appropriate FusionAuth Identity Provider with the same values (client_id, etc) as the original user management system you are migrating away from.

Import users with the Import API, assigning each user with a social login a random password such as a UUID.

Your next step depends on whether the social login provider’s unique identifier is available as part of your migration data. If you have the social login provider’s unique identifier, for each user, use the Link API to create a link with the appropriate User Id, Identity Provider Id and Identity Provider User Id.

• The User Id is the Id of the recently created FusionAuth User.

• The Identity Provider Id is found on the corresponding Identity Provider API documentation. Look for identityProvider.id.

• The Identity Provider User Id is the existing social provider user identifier exported or otherwise extracted from the original system.

You do not need to migrate the social network token, which may or may not be accessible. During the first login of a newly migrated user, FusionAuth finds the unique user in the social login provider based on the migrated Identity Provider User Id, and completes the login. During this process, FusionAuth stores a token on the Link, if the social provider returns one. Depending on the configuration of the social provider, users may see a prompt asking if they want to allow FusionAuth to have access to user data such as email address.

If you do not have the social login provider’s identifier, you need to decide if you want to transparently link the two accounts, which is easier for the end user, or if you want to ask the user to manually link the accounts, which is more accurate, but may be confusing.

To transparently link the accounts, choose a linking strategy of Link On Email or Link On Username, which will create the user if they don’t exist. However, if the user has an email address at their social provider which differs from the email address that was used to sign up for your application and which you imported to FusionAuth, then two accounts will be created.

For example, if the user has a Google account richard@gmail.com, but signed up for your application with richard@fusionauth.io, then if you use the Link On Email strategy, two different accounts will be created, since FusionAuth is trying to match on email address and they don’t. The same holds true for usernames with the Link on Username strategy.

To prompt the user to link the accounts, choose a linking strategy of Pending, which will prompt the end user to sign into FusionAuth after they sign into the social provider, authoritatively linking the two accounts.

Here’s more information about IdP Linking Strategies.

Other Entities

There are often other important entities, such as identity providers, clients, or keys, that need to be migrated. There are usually fewer of these, so an automated migration may not make sense, but plan to move this configuration somehow.

Be aware that functionality may not be the same between Keycloak and FusionAuth. This is different from user data; as long as you can somehow migrate a login identifier (a username or email) and a password hash, a user will be authenticated and successfully migrated. You can download FusionAuth before you begin a migration and build a proof of concept to learn more about the differences.

A partial list of what may need to be migrated for your application to work properly includes the following:

• In Keycloak, Identity Providers and User Federation allow user data to remain in external systems of record. FusionAuth has similar concepts of Identity Providers and Connectors.

• Mappers are ways for you to customize authentication or authorization workflows. FusionAuth has a similar concept called Lambdas.

• Keycloak has the ability to add custom scopes. FusionAuth does not, though there is an open feature request.

• With Keycloak, Clients are what your users can log in to. FusionAuth refers to these as Applications. In both cases, you can use either OAuth/OIDC or SAML to do so.

• Realms are a high level construct which groups other entities such as users and clients together. FusionAuth calls these Tenants. FusionAuth supports a multi-tenant configuration by default.

• For Keycloak, Roles provide information about what your users can do in your custom or off the shelf applications. Roles can be associated with other roles. FusionAuth has Roles and they are defined on an Application by Application basis and cannot be composed.

• Refresh tokens allow JWTs to be refreshed without a user logging in. These can be migrated using the Import Refresh Tokens API. Keycloak calls these "Offline Sessions"

Exporting From MySQL

Here’s the SQL to export from MySQL.

select FIRST_NAME, LAST_NAME, EMAIL, USERNAME, EMAIL_VERIFIED, U.ID as ID,
SECRET_DATA ->> '$.value' as PASSWORD,
SECRET_DATA ->> '$.salt' as SALT,
CREDENTIAL_DATA ->> '$.hashIterations' as HASHITERATIONS,
CREDENTIAL_DATA ->> '$.algorithm' as ALGORITHM,
CREATED_TIMESTAMP, REALM_ID
from USER_ENTITY U, CREDENTIAL C
where U.ID = C.USER_ID AND U.REALM_ID = 'RealmID'

You’ll need to run this SQL query in such a way as to get comma separated values (CSV) out of it. Depending on your database, you’ll do this differently.

Here’s how to generate CSV with MySQL, assuming the SQL above is in keycloak-export.sql. You may have to remove the header line from the out.csv file.

cat keycloak-export.sql | mysql -u USER -p keycloak| sed 's/\t/,/g' > out.csv

Exporting From PostgreSQL

Here’s the SQL to export from PostgreSQL.

select FIRST_NAME AS "FIRST_NAME", LAST_NAME AS "LAST_NAME", EMAIL AS "EMAIL",
USERNAME AS "USERNAME", EMAIL_VERIFIED AS "EMAIL_VERIFIED", U.ID AS "ID",
C.SECRET_DATA::jsonb ->> 'value' as "PASSWORD",
C.SECRET_DATA::jsonb ->> 'salt' as "SALT",
C.CREDENTIAL_DATA::jsonb ->> 'hashIterations' as "HASHITERATIONS",
C.CREDENTIAL_DATA::jsonb ->> 'algorithm' as "ALGORITHM",
CREATED_TIMESTAMP AS "CREATED_TIMESTAMP", REALM_ID AS "REALM_ID"
from USER_ENTITY U, CREDENTIAL C
where U.ID = C.USER_ID AND U.REALM_ID = 'RealmID'

Here’s how to generate CSV with PostgreSQL, assuming the SQL above is in keycloak-export-postgres.sql. You may have to remove the header line from the out.csv file.

psql -W -h localhost -p5433 -U USER -d keycloak -c "Copy (`cat keycloak-export-postgres.sql`) To STDOUT With CSV HEADER DELIMITER ',';" > out.csv

Export Result

Whichever database you are exporting from, at the end of the export process you’ll have a file that looks like:

FIRST_NAME,LAST_NAME,EMAIL,USERNAME,EMAIL_VERIFIED,ID,PASSWORD,SALT,HASHITERATIONS,ALGORITHM,CREATED_TIMESTAMP,REALM_ID
Test,Example,test@example.com,test,\0,f35a58e2-0247-4c38-aa39-93405e09c677,T6S/56cQy0ahQKohXe61aMOhvFr/PHEPfQbILKMLZKrdfOSo8wc+S6HCYomSJwTgYmdPy2gKh+oQW9UbeCmEwQ==,eYcTxcZhBV+GU9BQRt8Ypw==,27500,pbkdf2-sha256,1634670076567,Test
Test,Example2,test2@example.com,test2,
,1709a278-12a5-4126-9542-02f6809a349e,LjFqvhPuUHJdQvWIwVQfqxjeujAWqG/DVQRFoOv62/cTznl9ob4jwWwY6i1RrwGviu5iNPU5VIp03SxDyetyfw==,jVqbuA9k2Mlo37OWXBMKLw==,27500,pbkdf2-sha256,1634670197972,Test

Now, you can begin the user import process.

Set Up FusionAuth

You need to set up FusionAuth so migrated user data can be stored. As mentioned above, this guide assumes you have FusionAuth installed.

If you don’t, view our installation guides and install it before proceeding further.

Create a Test Tenant

It is best to create a separate tenant for migration testing. Tenants logically isolate configuration settings and users. If a migration goes awry or you need to redo it after tweaking settings, you can delete the test tenant and start with a clean system. To add a tenant, navigate to Tenants and choose the Add button (green plus sign).

Give it a descriptive Name like Keycloak import test. You shouldn’t need to modify any of the other configuration options to test importing users.

Save the tenant.

Record the Id of the tenant, which will be a UUID. It will look something like 25c9d123-8a79-4edd-9f76-8dd9c806b0f3. You’ll use this later.

Create a Test Application

Applications are anything a user can log in to. In FusionAuth there’s no differentiation between web applications, SaaS applications, APIs and native apps. To add an application, navigate to Applications and click on the Add button (the green plus sign). Give the application a descriptive name like Keycloak application.

Select your new tenant, created above, in the dropdown for the Tenant field.

Navigate to the OAuth tab and add an entry to Authorized redirect URLs. Use a dummy value such as https://fusionauth.io. Later, you’ll need to update this to be a valid redirect URL that can take the authorization code and exchange it for a token. Learn more about this in the FusionAuth OAuth documentation.

You shouldn’t need to modify any of the other configuration options to test importing users. Save the application.

Next, view the application by clicking the green magnifying glass and note the OAuth IdP login URL. You’ll be using it to test that users can log in.

Add an API Key

The next step is to create an API key. This will be used by the import script. To do so, navigate to Settings → API Keys in the administrative user interface.

This key needs to have the permission to run a bulk import of users. In the spirit of the principle of least privilege, give it the permission to POST to the /api/user/import endpoint. Record the API key string, as you’ll use it below.

Get the Script

FusionAuth provides an import script under a permissive open source license. It requires ruby (tested with ruby 2.7). To get the script, clone the git repository:

git clone https://github.com/FusionAuth/fusionauth-import-scripts

Navigate to the keycloak directory:

cd fusionauth-import-scripts/keycloak

Install Needed Gems

The following gems must be available to the import script:

• date

• csv

• optargs

• fusionauth_client

Most likely all of these will be on your system already, except the fusionauth_client gem.

If you have bundler installed, run bundle install in the keycloak directory. Otherwise install the needed gems in some other way.

Use the Script

You can see the output of the script by running it with the -h option:

ruby ./import.rb -h

The output will be similar to this:

Usage: import.rb [options]
    -r APPLICATION_IDS,              A comma separated list of existing applications Ids. All users will be registered for these applications.
        --register-users
    -u, --users-file USERS_FILE      The exported CSV user data file from Keycloak. Defaults to out.csv.
    -f FUSIONAUTH_URL,               The location of the FusionAuth instance. Defaults to http://localhost:9011.
        --fusionauth-url
    -k, --fusionauth-api-key API_KEY The FusionAuth API key.
    -t TENANT_ID,                    The FusionAuth tenant id. Required if more than one tenant exists.
        --fusionauth-tenant-id
    -m, --map-keycloak-user-id       Whether to map the keycloak user id for normal imported users to the FusionAuth user id.
    -h, --help                       Prints this help.

For this script to work correctly, set the following switches, unless the defaults work for you:

• -u should point to the location of the user export file you created.

• -f must point to your FusionAuth instance. If you are testing locally, it will probably be http://localhost:9011.

• -k needs to be set to the value of the API key created above.

• -t should be set to the Id of the testing tenant created above.

You may or may not want to use the -m switch, which takes the Keycloak Id and uses that for the FusionAuth user Id. If you have external systems reliant on the identifier, set this. Doing so ensures imported users have the same Id as they did in Keycloak. Otherwise, you can omit this switch.

When you run the script, you’ll see output like:

$ ruby ./import.rb -f http://localhost:9011 -k '...' -u out.csv
FusionAuth Importer : Keycloak
 > User file: out.csv
 > Call FusionAuth to import users
 > Import success
Duplicate users 0
Import complete. 2 users imported.

Verify the Import

Next, log in to the FusionAuth administrative user interface. Review the user entries to ensure the data was correctly imported.

You can manage the user by clicking on the Manage button (black button) to the right of the Created date in the list to review the details of the imported user’s profile.

If you have a test user whose password you know, open an incognito window and log in to ensure the hash migration was successful. You recorded the URL to log in to the example application in Create a Test Application.

After the test login, the user will be redirected to a URL like https://fusionauth.io/?code=FlZF97WIYLNxt4SGD_22qvpRh4fZ6kg_N89ZbBAy1E4&locale=fr&userState=Authenticated. This happens because you haven’t set up a web application to handle the authorization code redirect.

That is an important next step but is beyond the scope of this document. Consult the 5 minute setup guide for an example of how to do this.

The Final Destination of Imported Users

After you are done testing, you can choose to import users into the default tenant or a new tenant. Whichever you choose, make sure to update the -t switch to the correct value before running the import for the final time.

If you aren’t keeping users in the test tenant, delete it.

If you need to start over because the import failed or you need to tweak a setting, delete the tenant you created. This will remove all the users and other configurations for this tenant, giving you a fresh start. To do so, navigate to Tenants and choose the Delete button (red trash can icon).

Confirm your desire to delete the tenant. Depending on how many users you have imported, this may take some time.