"A request to the search index has failed" - OpenSearch(AWS)
-
Hello,
We're experiencing a known compatibility issue between FusionAuth and AWS-managed OpenSearch related to Point-in-Time (PIT) operations during bulk delete operations.
Issue Summary:
Error: SearchEngineRequestFailedException with message "A request to the search index has failed" Log Pattern: Unable to create point in time on index [fusionauth_user] Frequency: Occurs during nightly dedupe operations Impact: Prevents deletion of users via the bulk delete API Status Code: 503Environment:
FusionAuth Version: 1.55.1 Search Engine: AWS-managed OpenSearch Infrastructure: AWSBackground:
We understand this is a known issue with PIT operations against AWS OpenSearch, as documented in FusionAuth community discussions. We've reviewed release notes through version 1.62.1 and haven't found a specific fix for this issue.Questions:
- Is there a timeline for a fix to address AWS OpenSearch PIT compatibility?
- Are there alternative deletion methods that bypass PIT operations we could use as a workaround?
We'd appreciate any guidance on addressing this issue or information about planned fixes.
Thank you for your assistance.
-
@tim-clark Can you please point to the community discussion where this comes up? I could not find it in the issues.
-
@mark-robustelli Hi Mark,
Apologies for the delayed response. We've done some detailed investigation and can share concrete data.
Environment:
- FusionAuth 1.68.0
- search.type=elasticsearch (per your docs, the correct setting for OpenSearch)
- AWS-managed OpenSearch 2.19 (single t3.medium.search node)
- This issue has persisted from at least version 1.55.1 (Dec 2025) through to current 1.68.0
What we observe:
Our deletion job calls the bulk delete API once per user:
DELETE /api/user/bulk?queryString=<email>&hardDelete=true&dryRun=falseOn a typical night, 70–210 of these calls are made in rapid succession. A subset fail with:
ERROR io.fusionauth.api.service.search.BaseElasticsearchSearchEngine - Failed to set a Point in Time on index [fusionauth_user]
The response our application receives:
{
"generalErrors": [{"code": "[SearchEngineRequestFailedException]", "message": "A request to the search index has failed. This error is unexpected. Please review the troubleshooting guide..."}],
"status": 503
}CloudWatch metrics from our OpenSearch domain at the exact moment of failure (29 Jul, 23:12 UTC):
│ Metric │ Value │ │ JVM memory pressure │ 41% │ │ CPU utilisation │ 11% │ │ Thread pool search rejections │ 0 │ │ Thread pool write rejections │ 0 │ │ OpenSearch 5xx responses │ 0 │ │ OpenSearch 4xx responses │ elevated (spike correlates with deletion burst) │ │ HasUsedPointInTime │ 0 (PIT never successfully created) │ │ Request volume │ ~1,130 in 60 seconds vs normal ~15/min |The OpenSearch node is healthy and not under resource pressure. It appears to be actively rejecting the PIT creation request as a client error (4xx), not failing due to overload. The HasUsedPointInTime CloudWatch metric remains 0 throughout, confirming no PIT is ever successfully established.
We don't have visibility into the raw 4xx response body from OpenSearch — FusionAuth wraps it in the generic SearchEngineRequestFailedException. Our OpenSearch application logs only capture JVM-level warnings, and audit logging isn't enabled.
Questions:
- What PIT API endpoint does FusionAuth call internally when processing a queryString-based bulk delete? (e.g. POST /{index}/_pit vs POST /{index}/_search/point_in_time)
- Is it possible to expose the underlying search engine error response in the SearchEngineRequestFailedException? That would help diagnose whether this is a request format issue, a permissions issue,
or something else specific to AWS-managed OpenSearch. - Does FusionAuth use PIT when deleting by explicit userId array rather than queryString? If not, that might give us a workaround?
Thanks,
Tim