Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Better safeguard when deleting outdated query insights indices #261

Closed
ansjcy opened this issue Mar 10, 2025 · 0 comments · Fixed by #262
Closed

[FEATURE] Better safeguard when deleting outdated query insights indices #261

ansjcy opened this issue Mar 10, 2025 · 0 comments · Fixed by #262
Assignees
Labels
enhancement New feature or request

Comments

@ansjcy
Copy link
Member

ansjcy commented Mar 10, 2025

Is your feature request related to a problem?

The current implementation uses the isTopQueriesIndex function to filter indices when deleting outdated data. While this approach works currently, it presents a potential risk as the codebase evolves. If we modify index settings or patterns and update this function without careful consideration, we might inadvertently delete data from unrelated indices, as we currently iterate through all indices in the cluster.

What solution would you like?

Replace the broad clusterService.state().metadata().indices() approach to get delete candidates with a more targeted method that only considers indices matching our specific pattern, something like:

 ClusterStateRequest clusterStateRequest = new ClusterStateRequest()
            .clear()
            .indices(indexPattern)
            .metadata(true)
            .local(true)
            .indicesOptions(IndicesOptions.strictExpand());

This can limit deletion candidates exclusively to indices matching the top queries pattern, creating a safer boundary for our deletion operations.

What alternatives have you considered?

We could maintain the current implementation, but this would perpetuate the risk of potential data loss as the software evolves, particularly during refactoring or feature expansion.

Do you have any additional context?

This change follows the principle of least privilege by limiting the scope of our deletion operations to only the indices we intend to manage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants