I was able to do this by:
copying the advanced theme
editing it
navigating to the Helpers file
inside that, going to the head freemarker macro
I removed these lines:
<link rel="stylesheet" href="/css/font-awesome-4.7.0.min.css"/>
<link rel="stylesheet" href="/css/fusionauth-style.css?version=${version}"/>
[#-- Theme Stylesheet, only Authorize defines this boolean.
Using the ?no_esc on the stylesheet to allow selectors that contain a > symbols.
Once insde of a style tag we are safe and the stylesheet is validated not to contain an end style tag --]
[#if !(bypassTheme!false)]
<style>
${theme.stylesheet()?no_esc}
</style>
[/#if]
If you want to use the CSS field in the theme to store your CSs, leave in
[#if !(bypassTheme!false)]
<style>
${theme.stylesheet()?no_esc}
</style>
[/#if]
which pulls in the CSS associated with the theme. Doing so allows you to add all your custom CSS to the CSS field in the theme and have it automatically included. If you don't do this, make sure you add your CSS to this macro (either directly or via a link). You'll want to make sure you keep it in as you upgrade.
Removing all the CSS will result in a page that looks like the one below.
unstyled login page