There are a couple of things to check:

make sure that you've updated the issuer at the tenant screen: make sure you are using an asymmetric keypair to sign the id token. If you are using HMAC, which is the default for FusionAuth, you have to share a secret. Asymmetric algorithms like RSA256 are what proxies typically need (so they don't have to have the signing secret). More here: and here:

Hope that helps.