@mark-robustelli Hi Mark,
Apologies for the delayed response. We've done some detailed investigation and can share concrete data.
Environment:
FusionAuth 1.68.0
search.type=elasticsearch (per your docs, the correct setting for OpenSearch)
AWS-managed OpenSearch 2.19 (single t3.medium.search node)
This issue has persisted from at least version 1.55.1 (Dec 2025) through to current 1.68.0
What we observe:
Our deletion job calls the bulk delete API once per user:
DELETE /api/user/bulk?queryString=<email>&hardDelete=true&dryRun=false
On a typical night, 70–210 of these calls are made in rapid succession. A subset fail with:
ERROR io.fusionauth.api.service.search.BaseElasticsearchSearchEngine - Failed to set a Point in Time on index [fusionauth_user]
The response our application receives:
{
"generalErrors": [{"code": "[SearchEngineRequestFailedException]", "message": "A request to the search index has failed. This error is unexpected. Please review the troubleshooting guide..."}],
"status": 503
}
CloudWatch metrics from our OpenSearch domain at the exact moment of failure (29 Jul, 23:12 UTC):
│ Metric │ Value │
│ JVM memory pressure │ 41% │
│ CPU utilisation │ 11% │
│ Thread pool search rejections │ 0 │
│ Thread pool write rejections │ 0 │
│ OpenSearch 5xx responses │ 0 │
│ OpenSearch 4xx responses │ elevated (spike correlates with deletion burst) │
│ HasUsedPointInTime │ 0 (PIT never successfully created) │
│ Request volume │ ~1,130 in 60 seconds vs normal ~15/min |
The OpenSearch node is healthy and not under resource pressure. It appears to be actively rejecting the PIT creation request as a client error (4xx), not failing due to overload. The HasUsedPointInTime CloudWatch metric remains 0 throughout, confirming no PIT is ever successfully established.
We don't have visibility into the raw 4xx response body from OpenSearch — FusionAuth wraps it in the generic SearchEngineRequestFailedException. Our OpenSearch application logs only capture JVM-level warnings, and audit logging isn't enabled.
Questions:
What PIT API endpoint does FusionAuth call internally when processing a queryString-based bulk delete? (e.g. POST /{index}/_pit vs POST /{index}/_search/point_in_time)
Is it possible to expose the underlying search engine error response in the SearchEngineRequestFailedException? That would help diagnose whether this is a request format issue, a permissions issue,
or something else specific to AWS-managed OpenSearch.
Does FusionAuth use PIT when deleting by explicit userId array rather than queryString? If not, that might give us a workaround?
Thanks,
Tim
