Skip to content

Commit 7d42c3d

Browse files
committed
Remote state publication
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
1 parent ce18c28 commit 7d42c3d

File tree

1 file changed

+35
-3
lines changed

1 file changed

+35
-3
lines changed

_tuning-your-cluster/availability-and-recovery/remote-store/remote-cluster-state.md

+35-3
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,44 @@ In addition to the mandatory static settings, you can configure the following dy
5454
5555
Setting | Default | Description
5656
:--- | :--- | :---
57-
`cluster.remote_store.state.index_metadata.upload_timeout` | 20s | The amount of time to wait for index metadata upload to complete. Note that index metadata for separate indexes is uploaded in parallel.
58-
`cluster.remote_store.state.global_metadata.upload_timeout` | 20s | The amount of time to wait for global metadata upload to complete. Global metadata contains globally applicable metadata, such as templates, cluster settings, data stream metadata, and repository metadata.
59-
`cluster.remote_store.state.metadata_manifest.upload_timeout` | 20s | The amount of time to wait for the manifest file upload to complete. The manifest file contains the details of each of the files uploaded for a single cluster state, both index metadata files and global metadata files.
57+
`cluster.remote_store.state.index_metadata.upload_timeout` | 20s | Deprecated. Use `cluster.remote_store.state.global_metadata.upload_timeout` instead.
58+
`cluster.remote_store.state.global_metadata.upload_timeout` | 20s | The amount of time to wait for cluster state upload to complete.
59+
`cluster.remote_store.state.metadata_manifest.upload_timeout` | 20s | The amount of time to wait for the manifest file upload to complete. The manifest file contains the details of each of the files uploaded for a single cluster state, both index metadata files and global metadata files.
60+
`cluster.remote_store.state.cleanup_interval` | 300s | The interval for remote state clean-up async task to run. This task deletes the old remote state files.
6061

6162

6263
## Limitations
6364

6465
The remote cluster state functionality has the following limitations:
6566
- Unsafe bootstrap scripts cannot be run when the remote cluster state is enabled. When a majority of cluster-manager nodes are lost and the cluster goes down, the user needs to replace any remaining cluster manager nodes and reseed the nodes in order to bootstrap a new cluster.
67+
68+
## Remote Cluster State Publication
69+
The cluster state published to remote-backed storage can be used for publication. Currently, the active cluster manager
70+
sends the cluster state object over the transport layer to the follower nodes. This flow can be changed to fetch the
71+
cluster state from remote store. This can be done by enabling the experimental remote publication feature.
72+
Enable the feature flag for `remote_store.publication` feature by following the [experiment feature flag documentation]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/experimental/).
73+
When remote publication is enabled, the cluster manager node uploads the cluster state to remote store and then sends the
74+
remote path of the cluster state to the follower nodes. The follower nodes then download the cluster state from remote store.
75+
76+
The routing table is an object within the cluster state which contains the shard allocation details for each index.
77+
This object can become large in case of large number of shards in the cluster. Routing table is required to be stored in
78+
remote store for the remote publication to work. In order to enable remote persistence of routing table, the repository must
79+
be configured as below:
80+
81+
```yml
82+
# Remote routing table repository settings
83+
node.attr.remote_store.routing_table.repository: my-remote-routing-table-repo
84+
node.attr.remote_store.repository.my-remote-routing-table-repo.type: s3
85+
node.attr.remote_store.repository.my-remote-routing-table-repo.settings.bucket: <Bucket Name 3>
86+
node.attr.remote_store.repository.my-remote-routing-table-repo.settings.region: <Bucket region>
87+
```
88+
You do not have to use different remote store repositories for state and routing.
89+
These stores can share the same repository.
90+
91+
The relevant cluster settings for remote publication are listed below:
92+
93+
Setting | Default | Description
94+
:--- | :--- | :---
95+
`cluster.remote_store.state.read_timeout` | 20s | The amount of time to wait for remote state download to complete on the follower node.
96+
`cluster.remote_store.routing_table.path_type` | HASHED_PREFIX | Path type to be used for creating index routing path in blob store. Valid values are "FIXED", "HASHED_PREFIX", "HASHED_INFIX"
97+
`cluster.remote_store.routing_table.path_hash_algo` | FNV_1A_BASE64 | Algorithm to be used for constructing prefix or infix of blob store path. This setting comes into effect into if cluster.remote_store.routing_table.path_type is "hashed_prefix" or "hashed_infix". Valid values of algo are "FNV_1A_BASE64" or "FNV_1A_COMPOSITE_1"

0 commit comments

Comments
 (0)