-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Proposal] Writable Warm Status API Model #14640
Comments
@andrross @dblock @mch2 @ankitkala @rayshrey Any feedback on this would be greatly appreciated, thanks! |
Thanks @e-emoto! 2 quick questions from me:
Looking forward to any PRs! |
Hi @jed326, thanks for your response. |
Overall the API as
|
@e-emoto @dblock I think we can align this status API similar to recovery API which shows different index shard recovery status. There are 2 variants: a)
I think we can remove the
That is a good question. Index is a logical entity and we are tiering the data but in the unit of index, hence we are keeping it generic like |
thanks @e-emoto. Couple of other suggestions for the status API-
|
Is your feature request related to a problem? Please describe
The Status API in Writable Warm will be for listing the in-progress and failed index tierings. Since Writable Warm will eventually support other types of tierings for both dedicated and non-dedicated warm node clusters, the API needs to be extensible to cover those cases. The part of the design below describes the API model for the Status API, and the design for the rest of the API will be part of a follow-up task once some details about tiering metadata are figured out.
Describe the solution you'd like
API Model:
The API will use a source and target as input to filter which tierings are shown. It will validate that both inputs are valid tiers, and then use them to find any tierings that match the described type. The API should still work if only one of the source or target is given, and will find any tierings with that input, allowing for more flexible queries. In the default case if no source or target is given as input, the status API should return all in progress or failed tierings for the specified indices, regardless of the tiering change.
API Request:
GET /<indexNameOrPattern>/_tier?source=hot&target=warm
GET /<indexNameOrPattern>/_tier?status=ongoing
GET /<indexNameOrPattern>/_tier?verbose=true/false
The API would be a get request that has a few parameters. The index name in the path will be required, but can support using ‘
_all
’ or ‘*
’ to get migrations from all indices that match the parameters. The API will also support comma separated index names.API Parameters:
source = hot / warm
(optional, no default)target = hot / warm
(optional, no default)The values for the
source
andtarget
parameters are the tiers, withsource
being the tier the index started in andtarget
being the tier it is moving to.status = failed / ongoing
(optional, no default)The values of the
status
parameter represent the state of the tiering.failed
indicates that the tiering has failed andongoing
means the tiering process is in progress.verbose = true / false
(default false)The
verbose
parameter determines whether the API response should include details like the shard relocation status, failure reason, and tiering start time.API Response:
Success:
With Verbose Flag:
Failure:
Example API Use Cases:
Get All Ongoing Tierings:
GET /_all/_tier?status=ongoing
Get All Failed Hot To Warm Tierings:
GET /_all/_tier?source=hot&target=warm&status=failed
Get Shard Details for a Specific Index Tiering:
GET /target_index/_tier?verbose=true
Related component
Search:Remote Search
Describe alternatives you've considered
No response
Additional context
This issue is for getting feedback on the API structure, and will be followed up with a PR for the API spec and a low level design description.
Related issues:
#14679
#13294
The text was updated successfully, but these errors were encountered: