[Feature Request] Migrate Update by query, delete by query, reindex to use PiT instead of scroll

[shwetathareja]

Is your feature request related to a problem? Please describe

Today, update by query, delete by query and reindex uses scroll API for searching.

Describe the solution you'd like

We should move these APIs to using Point in Time instead of scroll. It is now the preferred pagination for OpenSearch.

Related component

Search

Describe alternatives you've considered

No response

Additional context

No response

[peternied]

[Triage - attendees 1 2 3 4 5]
@shwetathareja Thanks for creating this issue, this would be a good improvement. Note; I've tagged this as breaking since functionality between scroll and PIT based systems might be different. We should evaluate these scenarios before considering backporting.

[kkewwei]

@shwetathareja If i can try it if you haven't do? I was planning to run those operations concurrently #14516, but it seems depend on the pr, Can I also pick up this?

[cwperks]

@kkewwei If we change the internal implementation, can you also wrap in a privileged block like is done on this PR? #17250

If the internals aren't wrapped, then users may need to update roles definitions when using the security plugin. As long as the user's have permission to use the respective action like Update by Query I think we shouldn't need to authorize any additional transport actions executed internally.

[kkewwei]

@kkewwei If we change the internal implementation, can you also wrap in a privileged block like is done on this PR? #17250

If the internals aren't wrapped, then users may need to update roles definitions when using the security plugin. As long as the user's have permission to use the respective action like Update by Query I think we shouldn't need to authorize any additional transport actions executed internally.

@cwperks Get it, thanks.

[kkewwei]

@shwetathareja @peternied @cwperks I understand that the advantages of pit are that, It can retry after failure and can query different data, but scroll cannot. Are there any other advantages you can add? Is this replacement a good enough reason?