Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add markers for cluster APIs. #9110

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8234ec2
Add markers for cluster APIs.
Naarcha-AWS Jan 24, 2025
9ac0957
Merge branch 'main' into cluster-apis
Naarcha-AWS Jan 24, 2025
7f0f39b
Add rest of cluster inserts
Naarcha-AWS Jan 24, 2025
44d9145
Merge branch 'main' into cluster-apis
Naarcha-AWS Jan 30, 2025
94b4cd1
Adjust tables. Add example requests and responses.
Naarcha-AWS Feb 4, 2025
eff3c78
Merge branch 'main' into cluster-apis
Naarcha-AWS Feb 4, 2025
645f9e1
Apply suggestions from code review
Naarcha-AWS Feb 4, 2025
b713b3b
Apply suggestions from code review
Naarcha-AWS Feb 4, 2025
bab9e3c
Merge branch 'main' into cluster-apis
Naarcha-AWS Feb 7, 2025
5607382
Merge branch 'main' into cluster-apis
Naarcha-AWS Feb 12, 2025
5f60253
Merge branch 'main' into cluster-apis
Naarcha-AWS Feb 14, 2025
7550e90
Remove depracated parameters.
Naarcha-AWS Feb 14, 2025
5fbb2c4
One last deprecated parameter
Naarcha-AWS Feb 14, 2025
19b60e4
Description additions.
Naarcha-AWS Feb 14, 2025
ecdce80
Merge branch 'main' into cluster-apis
Naarcha-AWS Feb 20, 2025
7c8ea81
Fix formatting error.
Naarcha-AWS Feb 20, 2025
662c507
Apply suggestions from code review
Naarcha-AWS Feb 21, 2025
403e330
Apply suggestions from code review
Naarcha-AWS Feb 21, 2025
999aef7
Apply suggestions from code review
Naarcha-AWS Feb 24, 2025
9ea5bc7
Apply suggestions from code review
Naarcha-AWS Feb 26, 2025
93cfb7b
Fix other marker items and other feedback
Naarcha-AWS Feb 27, 2025
96a153f
Fix code examples
Naarcha-AWS Mar 4, 2025
437d733
Merge branch 'main' into cluster-apis
Naarcha-AWS Mar 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 25 additions & 15 deletions _api-reference/cluster-api/cluster-allocation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,26 +12,36 @@ redirect_from:
**Introduced 1.0**
{: .label .label-purple }

The most basic cluster allocation explain request finds an unassigned shard and explains why it can't be allocated to a node.

If you add some options, you can instead get information on a specific shard, including why OpenSearch assigned it to its current node.
The Cluster Allocation Explain API explains how shards are allocated in the cluster and why unassigned shards cannot be assigned to a node.


<!-- spec_insert_start
api: cluster.allocation_explain
component: endpoints
-->
## Endpoints

```json
GET _cluster/allocation/explain
POST _cluster/allocation/explain
GET /_cluster/allocation/explain
POST /_cluster/allocation/explain
```
<!-- spec_insert_end -->


<!-- spec_insert_start
api: cluster.allocation_explain
columns: Parameter, Data type, Description, Default
component: query_parameters
-->
## Query parameters

All parameters are optional.
The following table lists the available query parameters. All query parameters are optional.

Parameter | Type | Description
:--- | :--- | :---
include_yes_decisions | Boolean | OpenSearch makes a series of yes or no decisions when trying to allocate a shard to a node. If this parameter is true, OpenSearch includes the (generally more numerous) "yes" decisions in its response. Default is `false`.
include_disk_info | Boolean | Whether to include information about disk usage in the response. Default is `false`.
| Parameter | Data type | Description | Default |
| :--- | :--- | :--- | :--- |
| `include_disk_info` | Boolean | When `true`, returns information about disk usage and shard sizes. | `false` |
| `include_yes_decisions` | Boolean | When `true`, returns any `YES` decisions in the allocation explanation. | `false` |

<!-- spec_insert_end -->


## Request body fields
Expand All @@ -40,10 +50,10 @@ All cluster allocation explain fields are optional.

Field | Type | Description
:--- | :--- | :---
current_node | String | If you only want an explanation if the shard happens to be on a particular node, specify that node name here.
index | String | The name of the shard's index.
primary | Boolean | Whether to provide an explanation for the primary shard (true) or its first replica (false), which share the same shard ID.
shard | Integer | The shard ID that you want an explanation for.
`current_node` | String | Specifies a node name to return an explanation only if the shard is on that node.
`index` | String | The name of the shard's index.
`primary` | Boolean | Whether to provide an explanation for the primary shard (`true`) or its first replica (`false`), both of which share the same shard ID.
`shard` | Integer | The shard ID that you want an explanation for.

## Example request

Expand Down
54 changes: 45 additions & 9 deletions _api-reference/cluster-api/cluster-awareness.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,31 +15,66 @@ redirect_from:

To control the distribution of search or HTTP traffic, you can use the weights per awareness attribute to control the distribution of search or HTTP traffic across zones. This is commonly used for zonal deployments, heterogeneous instances, and routing traffic away from zones during zonal failure.

<!-- spec_insert_start
api: cluster.get_weighted_routing
component: endpoints
-->
## Endpoints
```json
GET /_cluster/routing/awareness/{attribute}/weights
```
<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_weighted_routing
component: endpoints
omit_header: true
-->
```json
PUT /_cluster/routing/awareness/<attribute>/weights
GET /_cluster/routing/awareness/<attribute>/weights?local
GET /_cluster/routing/awareness/<attribute>/weights
PUT /_cluster/routing/awareness/{attribute}/weights
```
<!-- spec_insert_end -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there is a path parameter attribute. Please include the path parameters section.


<!-- spec_insert_start
api: cluster.put_weighted_routing
component: path_parameters
-->
## Path parameters

Parameter | Type | Description
:--- | :--- | :---
attribute | String | The name of the awareness attribute, usually `zone`. The attribute name must match the values listed in the request body when assigning weights to zones.
The following table lists the available path parameters.

| Parameter | Required | Data type | Description |
| :--- | :--- | :--- | :--- |
| `attribute` | **Required** | String | The name of the awareness attribute. |

<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_weighted_routing
columns: Parameter, Data type, Description, Default
component: query_parameters
-->
## Query parameters

The following table lists the available query parameters. All query parameters are optional.

| Parameter | Data type | Description | Default |
| :--- | :--- | :--- | :--- |

<!-- spec_insert_end -->


## Request body fields

Parameter | Type | Description
:--- | :--- | :---
weights | JSON object | Assigns weights to attributes within the request body of the PUT request. Weights can be set in any ratio, for example, 2:3:5. In a 2:3:5 ratio with 3 zones, for every 100 requests sent to the cluster, each zone would receive either 20, 30, or 50 search requests in a random order. When assigned a weight of `0`, the zone does not receive any search traffic.
_version | String | Implements optimistic concurrency control (OCC) through versioning. The parameter uses simple versioning, such as `1`, and increments upward based on each subsequent modification. This allows any servers from which a request originates to validate whether or not a zone has been modified.
`weights` | JSON object | Assigns weights to attributes within the request body of the PUT request. Weights can be set in any ratio, for example, 2:3:5. In a 2:3:5 ratio with 3 zones, for every 100 requests sent to the cluster, each zone would receive either 20, 30, or 50 search requests in a random order. When assigned a weight of `0`, the zone does not receive any search traffic.
`_version` | String | Implements optimistic concurrency control (OCC) through versioning. The parameter uses simple versioning, such as `1`, and increments upward based on each subsequent modification. This allows any servers from which a request originates to validate whether or not a zone has been modified.


In the following example request body, `zone_1` and `zone_2` receive 50 requests each, whereas `zone_3` is prevented from receiving requests:

```
```json
{
"weights":
{
Expand Down Expand Up @@ -117,6 +152,7 @@ OpenSearch responds with the weight of each zone:
},
"_version":1
}
```


## Next steps
Expand Down
142 changes: 142 additions & 0 deletions _api-reference/cluster-api/cluster-component-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
---
layout: default
title: Cluster component template
nav_order: 25
parent: Cluster APIs
---

# Cluster component template
**Introduced 1.0**
{: .label .label-purple }

The Cluster Component Template API creates or updates the cluster's component templates.

# Endpoints

The `GET` method returns one or more component templates:

<!-- spec_insert_start
api: cluster.get_component_template
component: endpoints
omit_header: true
-->
```json
GET /_component_template
GET /_component_template/{name}
```
<!-- spec_insert_end -->

The `EXISTS` method returns information about whether a component template exists:

<!-- spec_insert_start
api: cluster.exists_component_template
component: endpoints
omit_header: true
-->
```json
HEAD /_component_template/{name}
```
<!-- spec_insert_end -->

The `PUT` method creates or updates a component template:

<!-- spec_insert_start
api: cluster.put_component_template
component: endpoints
omit_header: true
-->
```json
POST /_component_template/{name}
PUT /_component_template/{name}
```
<!-- spec_insert_end -->

The `DELETE` method deletes a component template:

<!-- spec_insert_start
api: cluster.delete_component_template
component: endpoints
omit_header: true
-->
```json
DELETE /_component_template/{name}
```
<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_component_template
component: path_parameters
-->
## Path parameters

The following table lists the available path parameters.

| Parameter | Required | Data type | Description |
| :--- | :--- | :--- | :--- |
| `name` | **Required** | String | The name of the component template to create. OpenSearch includes the following built-in component templates: `logs-mappings`; `logs-settings`; `metrics-mappings`; `metrics-settings`; `synthetics-mapping`; `synthetics-settings`. OpenSearch uses these templates to configure backing indexes for its data streams. If you want to overwrite one of these templates, set the replacement template `version` to a higher value than the current version. If you want to disable all built-in component and index templates, set `stack.templates.enabled` to `false` using the Cluster Update Settings API. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this list auto-generated? If so, we should change the generator to use commas instead of semicolons between list elements. cc: @nhtruong

Also, can we link to the cluster update settings API?

Copy link
Collaborator Author

@Naarcha-AWS Naarcha-AWS Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me check is the list is autogenerated, considering the list's placement in the description, I'm assuming no.


<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_component_template
columns: Parameter, Data type, Description, Default
include_deprecated: false
component: query_parameters
-->
## Query parameters

The following table lists the available query parameters. All query parameters are optional.

| Parameter | Data type | Description | Default |
| :--- | :--- | :--- | :--- |
| `cluster_manager_timeout` | String | A duration. Units can be `nanos`, `micros`, `ms` (milliseconds), `s` (seconds), `m` (minutes), `h` (hours) and `d` (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. | N/A |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several comments on cluster_manager_timeout:

  1. Data type should be "Time unit"
  2. This description is not descriptive. The one we have currently is better "The amount of time to wait for a response from the cluster manager node."
  3. For valid units, we can link to the data type to https://opensearch.org/docs/latest/api-reference/common-parameters/#time-units
  4. 0 and 1 should be in tic marks, not quotation marks
  5. The default is 30 seconds -> this needs to be updated in the spec.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Time unit" is not a supported data type, according to the list of supported Data Types for OpenAPI. https://swagger.io/docs/specification/v3_0/data-models/data-types/

Not sure if there's a way to change the Data type manually beyond editing the table directly. That being said, I wouldn't advise it, since "String" matches what's in the API spec.

| `create` | Boolean | When `true`, this request cannot replace or update existing component templates. | `false` |
| `timeout` | String | A duration. Units can be `nanos`, `micros`, `ms` (milliseconds), `s` (seconds), `m` (minutes), `h` (hours) and `d` (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. | N/A |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as cluster_manager_timeout.


<!-- spec_insert_end -->

## Example request

```json
POST /_component_template/logs_template
{
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"timestamp": {
"type": "date"
},
"message": {
"type": "text"
},
"level": {
"type": "keyword"
},
"service": {
"type": "keyword"
}
}
},
"aliases": {
"logs_alias": {}
}
},
"version": 1,
"meta": {
"description": "Component template for log data"
}
}
```
{% include copy-curl.html %}

## Example response

```json
{
"acknowledged": true
}
```
44 changes: 37 additions & 7 deletions _api-reference/cluster-api/cluster-decommission.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,55 @@ redirect_from:
**Introduced 1.0**
{: .label .label-purple }

The cluster decommission operation adds support decommissioning based on awareness. It greatly benefits multi-zone deployments, where awareness attributes, such as `zones`, can aid in applying new upgrades to a cluster in a controlled fashion. This is especially useful during outages, in which case, you can decommission the unhealthy zone to prevent replication requests from stalling and prevent your request backlog from becoming too large.
The Cluster Decommission API adds support decommissioning based on awareness. It greatly benefits multi-zone deployments, where awareness attributes, such as `zones`, can aid in applying new upgrades to a cluster in a controlled fashion. This is especially useful during outages, in which case, you can decommission the unhealthy zone to prevent replication requests from stalling and prevent your request backlog from becoming too large.

For more information about allocation awareness, see [Shard allocation awareness]({{site.url}}{{site.baseurl}}//opensearch/cluster/#shard-allocation-awareness).


<!-- spec_insert_start
api: cluster.get_decommission_awareness
component: endpoints
-->
## Endpoints
```json
GET /_cluster/decommission/awareness/{awareness_attribute_name}/_status
```
<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_decommission_awareness
component: endpoints
omit_header: true
-->
```json
PUT /_cluster/decommission/awareness/{awareness_attribute_name}/{awareness_attribute_value}
```
<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.delete_decommission_awareness
component: endpoints
omit_header: true
-->
```json
PUT /_cluster/decommission/awareness/{awareness_attribute_name}/{awareness_attribute_value}
GET /_cluster/decommission/awareness/{awareness_attribute_name}/_status
DELETE /_cluster/decommission/awareness
```
<!-- spec_insert_end -->

<!-- spec_insert_start
api: cluster.put_decommission_awareness
component: path_parameters
-->
## Path parameters

Parameter | Type | Description
:--- | :--- | :---
awareness_attribute_name | String | The name of awareness attribute, usually `zone`.
awareness_attribute_value | String | The value of the awareness attribute. For example, if you have shards allocated in two different zones, you can give each zone a value of `zone-a` or `zoneb`. The cluster decommission operation decommissions the zone listed in the method.
The following table lists the available path parameters.

| Parameter | Required | Data type | Description |
| :--- | :--- | :--- | :--- |
| `awareness_attribute_name` | **Required** | String | The name of the awareness attribute. |
| `awareness_attribute_value` | **Required** | String | The value of the awareness attribute. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old descriptions for both of these parameters were more descriptive. Could we propagate them to the api-spec repo?


<!-- spec_insert_end -->

## Example requests

Expand Down
Loading