@chakshu
Sorry, I pointed you to the incorrect setting.
You can go to Applications > FusionAuth > Edit > JWT > Refresh Token duration
Changing that to 1 (the value is in minutes) caused me to be signed out of the admin application after 60 seconds.
Hope that helps.