Unlimited .data fields
-
Hello,
This is not a direct bug, but maybe a safety net for other users to not do the same mistake as we did.
We had a prod environment going down due to fusionauth OOM errors which were cause by a bug on one of our services. There is everything fine with the FusionAuth in general, but the problem was that we filled up the users.data field for each user with too much data due to an error (should only have been a list of some bytes). Therefore, after some months we started to have some occasional OOM errors of FusionAuth, as the 0.5GB RAM were not sufficient anymore to load even a single user (which had a users.data text field of 400MB).
After cleaning that, everything is back to normal.
My proposition might be, to put a (maybe configurable) size limit on the *.data fields to prevent such hard to catch runtime errors.
Feel free to ask back for more info, I just wanted to put this here in case you might wanna consider it
-
@cyrill-lippuner Thanks for sharing this info with the community!
Did you also explore increasing the memory available to FusionAuth? That would be another approach, if someone actually really needed a large amount of data in
user.data.However, 400MB is a pretty big JSON blob!
If you'd like to file a feature request about limiting user.data size, that'd be awesome.
-
@dan Yes I did increase it until 2GB, but then loading a list of 4 users also fails ^^
So I think it is just not a good idea using FA as a database
Will look into the feature request.
-
@cyrill-lippuner Thanks for filing the feature request: https://github.com/FusionAuth/fusionauth-issues/issues/1901
-
@cyrill-lippuner said in Unlimited .data fields:
Hello,
This is not a direct bug, but maybe a safety net for other users to not do the same mistake as we did.
We had a prod environment going down due to fusionauth OOM errors which were cause by a bug on one of our services. There is everything fine with the FusionAuth in general, but the problem was that we filled up the users.data field for each user with too much data due to an error (should only have been a list of some bytes). Therefore, after some months we started to have some occasional OOM errors of FusionAuth, as the 0.5GB RAM were not sufficient anymore to load even a single user (which had a users.data text field of 400MB).
After cleaning that, everything is back to normal.
My proposition might be, to put a (maybe configurable) size limit on the *.data fields to prevent such hard to catch runtime errors.
Feel free to ask back for more info, I just wanted to put this here in case you might wanna consider itImplementing a configurable size limit on data fields is indeed a useful safety measure. This limit can help prevent unintended data growth and ensure that the system behaves predictably even when users inadvertently input excessive data.
