You can run FusionAuth in a docker container and spin it up using the docker install instructions in your CI/CD pipeline.

You may also be interested in using Kickstart, which lets you configure every aspect of FusionAuth (except adding multiple API keys) via a simple JSON file. This ensures your integration tests run against an auth system that has been configured exactly as you expect.