Skip to content

Commit d33d24e

Browse files
kaushalmahi12dependabot[bot]peteralfonsiPeter Alfonsiimvtsl
authored
Add changes to propagate queryGroupId across child requests and nodes (opensearch-project#14614)
* add query group header propagator Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * apply spotless check Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add new propagator in ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * spotlessApply Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * address comments Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * add PR link changelog Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635) * Allow system index warning Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Address code review comments Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Star tree codec changes (opensearch-project#14514) --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672) * Bump com.github.spullara.mustache.java:compiler Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14. - [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14) --- updated-dependencies: - dependency-name: com.github.spullara.mustache.java:compiler dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673) * Bump net.minidev:accessors-smart in /plugins/repository-azure Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1. - [Release notes](https://github.com/netplex/json-smart-v2/releases) - [Commits](netplex/json-smart-v2@2.5.0...2.5.1) --- updated-dependencies: - dependency-name: net.minidev:accessors-smart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> * move query group thread context propagator out of ThreadContext Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> --------- Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
1 parent d351c58 commit d33d24e

File tree

4 files changed

+97
-0
lines changed

4 files changed

+97
-0
lines changed

CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
1414
- Fix race condition while parsing derived fields from search definition ([14445](https://github.com/opensearch-project/OpenSearch/pull/14445))
1515
- Add `strict_allow_templates` dynamic mapping option ([#14555](https://github.com/opensearch-project/OpenSearch/pull/14555))
1616
- Add allowlist setting for ingest-common and search-pipeline-common processors ([#14439](https://github.com/opensearch-project/OpenSearch/issues/14439))
17+
- [Workload Management] add queryGroupId header propagator across requests and nodes ([#14614](https://github.com/opensearch-project/OpenSearch/pull/14614))
1718
- Create SystemIndexRegistry with helper method matchesSystemIndex ([#14415](https://github.com/opensearch-project/OpenSearch/pull/14415))
1819
- Print reason why parent task was cancelled ([#14604](https://github.com/opensearch-project/OpenSearch/issues/14604))
1920

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.wlm;
10+
11+
import org.opensearch.common.util.concurrent.ThreadContextStatePropagator;
12+
13+
import java.util.HashMap;
14+
import java.util.List;
15+
import java.util.Map;
16+
17+
/**
18+
* This class is used to propagate QueryGroup related headers to request and nodes
19+
*/
20+
public class QueryGroupThreadContextStatePropagator implements ThreadContextStatePropagator {
21+
// TODO: move this constant to QueryGroupService class once the QueryGroup monitoring framework PR is ready
22+
public static List<String> PROPAGATED_HEADERS = List.of("queryGroupId");
23+
24+
/**
25+
* @param source current context transient headers
26+
* @return the map of header and their values to be propagated across request threadContexts
27+
*/
28+
@Override
29+
@SuppressWarnings("removal")
30+
public Map<String, Object> transients(Map<String, Object> source) {
31+
final Map<String, Object> transientHeaders = new HashMap<>();
32+
33+
for (String headerName : PROPAGATED_HEADERS) {
34+
transientHeaders.compute(headerName, (k, v) -> source.get(headerName));
35+
}
36+
return transientHeaders;
37+
}
38+
39+
/**
40+
* @param source current context headers
41+
* @return map of header and their values to be propagated across nodes
42+
*/
43+
@Override
44+
@SuppressWarnings("removal")
45+
public Map<String, String> headers(Map<String, Object> source) {
46+
final Map<String, String> propagatedHeaders = new HashMap<>();
47+
48+
for (String headerName : PROPAGATED_HEADERS) {
49+
propagatedHeaders.compute(headerName, (k, v) -> (String) source.get(headerName));
50+
}
51+
return propagatedHeaders;
52+
}
53+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
/**
10+
* This package contains workload management constructs
11+
*/
12+
13+
package org.opensearch.wlm;
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.wlm;
10+
11+
import org.opensearch.test.OpenSearchTestCase;
12+
13+
import java.util.Map;
14+
15+
public class QueryGroupThreadContextStatePropagatorTests extends OpenSearchTestCase {
16+
17+
public void testTransients() {
18+
QueryGroupThreadContextStatePropagator sut = new QueryGroupThreadContextStatePropagator();
19+
Map<String, Object> source = Map.of("queryGroupId", "adgarja0r235te");
20+
Map<String, Object> transients = sut.transients(source);
21+
assertEquals("adgarja0r235te", transients.get("queryGroupId"));
22+
}
23+
24+
public void testHeaders() {
25+
QueryGroupThreadContextStatePropagator sut = new QueryGroupThreadContextStatePropagator();
26+
Map<String, Object> source = Map.of("queryGroupId", "adgarja0r235te");
27+
Map<String, String> headers = sut.headers(source);
28+
assertEquals("adgarja0r235te", headers.get("queryGroupId"));
29+
}
30+
}

0 commit comments

Comments
 (0)