I am using OpenSearch 2.13 with .NET 8 and trying to determine if an index pattern already exists before creating a new one. The issue is that OpenSearch allows multiple index patterns with the same name but different IDs when created via API.
Current Approach:
I am retrieving all the existing index patterns for a specific tenant using the following API:
var requestUri = utilityServices.GetUrl($"/api/saved_objects/_find?fields=title&fields=type&per_page=10000&type=index-pattern");
var headers = new Dictionary<string, string?>
{
{ "securitytenant", indexNameService.NormalizeIndexName(customerName) },
{ "osd-xsrf", "osd-fetch" }
};
After retrieving them, I iterate through the patterns to check if an index pattern with the same title exists.
var existingPatterns = existingIndexPattern?.saved_objects?.Select(s => (s.id, s.attributes.title, s.version)) ?? [];
foreach (var pattern in indexPatterns)
{
var existingPattern = existingPatterns.FirstOrDefault(pair => pair.title == pattern);
if (string.IsNullOrEmpty(existingPattern.id))
{
// Create a new index pattern
var requestUri = utilityServices.GetUrl($"/api/saved_objects/index-pattern");
var headers = new Dictionary<string, string?>
{
{ "securitytenant", indexNameService.NormalizeIndexName(customerName) },
{ "osd-xsrf", "osd-fetch" }
};
var requestBody = new
{
attributes = new
{
title = indexPattern,
timeFieldName = "version"
},
};
}
else
{
// Refreshing the mappings for an existing index pattern
// I have reverse-engineered the refresh field list functionality of index patterns from the OpenSearch dashboard
var requestUri = utilityServices.GetUrl($"/api/index_patterns/_fields_for_wildcard?pattern={indexPattern}");
var headers = new Dictionary<string, string>
{
{ "securitytenant", indexNameService.NormalizeIndexName(customerName) ?? "" },
{ "osd-xsrf", "osd-fetch" }
};
var response = await utilityServices.SendHttp<Dictionary<string, List<Field>>>(requestUri, HttpMethod.Get, headers: headers);
var fields = response.FirstOrDefault().Value;
// Updating the existing index pattern with the latest mappings
var updateRequestUri = utilityServices.GetUrl($"/api/saved_objects/index-pattern/{existingPattern.id}");
var requestBody = new IndexPatternPutRequest
{
Attributes = new DashboardAttributes
{
Title = existingPattern.title,
TimeFieldName = "version",
Fields = JsonSerializer.Serialize(fields) // Updating fields to refresh mappings
},
Version = existingPattern.version,
};
var json = JsonSerializer.Serialize(requestBody);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var updateResponse = await utilityServices.SendHttp<SavedObject>(updateRequestUri, HttpMethod.Put, headers: headers, content: content);
}
Issue:
Even with this logic, I am still encountering cases where multiple index patterns with the same title are created. This is not happening in all cases but occurs intermittently.
Question:
Is there a better way to check whether an index pattern already exists and update or refresh it with the latest mappings instead of creating duplicates? Is there an OpenSearch API that can handle this more efficiently? I can't find any support for this in OpenSearch's documentation.
Any guidance or alternative approaches would be greatly appreciated.
发布者:admin,转转请注明出处:http://www.yc00.com/questions/1744856793a4597454.html
评论列表(0条)