Will bulk user imports into one tenant affect performance in other tenants?
-
We are building a migration application that imports users from a legacy database into FusionAuth. Some clients have millions of users that need to be migrated.
Our migration process:
- Users are imported in batches (up to 100,000 per batch following the Import API recommendations)
- Passwords are sent pre-hashed with salt, factor, and encryption scheme
- Initial import phase may take up to 2 days to complete
- After initial import, ongoing sync uses the
/api/user/{userId}PATCH endpoint for updates - Duplicates are unlikely as data goes through cleanup before import
- We have Kafka integration enabled for events (all event toggles enabled per tenant)
- No transactional webhooks are configured
Our FusionAuth deployment is multi-tenant, and migrations need to run without interrupting day-to-day operations for other clients on the platform.
Question: If we initiate a bulk user import into one tenant, how will this affect activity on other tenants? Will users logging in or managing their data in other tenants experience performance issues?
-
FusionAuth tenants provide logical isolation, not physical isolation. While users, applications, groups, and configuration are logically separated per tenant, all tenants in an instance share the same FusionAuth nodes, database, and search index.
This means that while importing data into one tenant will not impact the data integrity in another tenant, it does draw on the same compute and database capacity that other tenants' logins and profile updates use. Large imports can potentially affect performance across all tenants.
Recommendations to minimize impact:
-
Serialize imports: Do not run multiple imports simultaneously. Even when migrating multiple clients, avoid having two clients in the initial import phase at the same time.
-
Pre-hash passwords: Since you're already sending passwords with hash, salt, factor, and encryption scheme, this will significantly reduce load during imports. If passwords were sent as plaintext, FusionAuth would need to hash each one on import (which can take several hundred milliseconds per user).
-
Start with smaller batches: Rather than immediately using 100k user batches, start smaller to monitor system performance during imports. If performance is acceptable, gradually increase the batch size until you find the right balance between performance and import speed.
-
Avoid transactional webhooks: You mentioned using Kafka integration instead of custom webhooks, which is good. Ensure you are only using the Kafka integration for events and not using any transactional webhooks, as these can add additional load during imports. Transactional webhooks block the operation until the webhook completes, which can significantly slow down bulk imports. Non-transactional events (like those sent to Kafka) are fire-and-forget and don't block the import process.
-
Limit concurrent operations: You mentioned planning to run only one import call at a time from one worker, which is the right approach. Updates using the PATCH endpoint can occur in parallel as they should be infrequent.
-
Monitor database performance: Keep an eye on your database connection pool and query performance during imports. The shared database is typically the bottleneck in multi-tenant scenarios under heavy load.
By following these practices, you can minimize the impact of bulk imports on other tenants in your multi-tenant FusionAuth deployment.
Related Documentation
- Multi-tenancy Guide - Overview of FusionAuth's multi-tenant architecture and concepts
- Levels of Isolation - Explanation of logical vs. physical isolation in FusionAuth
- Import Users API - API documentation for bulk user imports
- Password Hashes - Details on importing pre-hashed passwords to improve performance
- Transactional vs. Non-Transactional Events - Understanding event types and their performance impact
- Kafka Integration - Using Kafka for non-blocking event processing
- Scaling Your Auth - Best Practices - Performance and scaling considerations for authentication systems
-
-
D dan has marked this topic as solved