Skip to content

Commit ba0df74

Browse files
akolarkunnuakolarkunnu
and
akolarkunnu
authored
COMPAT locale provider will be removed in a future release (opensearch-project#13988)
* COMPAT locale provider will be removed in a future release Description: From JDK21 onwards a new warning has started to come, "WARNING: COMPAT locale provider will be removed in a future release". So, we have to avoid usage of COMPAT provider. We were setting exlpicitly to COMPAT locale provider in couple of places, this change is to convert COMPAT to CLDR locale provider. After this change, couple of tests started to fail becasue some locale data has minor changes in CLDR compared to COMPAT. For example, day and month short names of GERMAN "de" locale are different in CLDR and COMPAT, just need to add a . in the end for CLDR. Resolves opensearch-project#11550 Signed-off-by: Muneer Kolarkunnu <muneer.kolarkunnu@netapp.com> Signed-off-by: akolarkunnu <abdul.kolarkunnu@netapp.com> * COMPAT locale provider will be removed in a future release Description: From JDK21 onwards a new warning has started to come, "WARNING: COMPAT locale provider will be removed in a future release". So, we have to avoid usage of COMPAT provider. We were setting exlpicitly to COMPAT locale provider in couple of places, this change is to convert COMPAT to CLDR locale provider. After this change, couple of tests started to fail becasue some locale data has minor changes in CLDR compared to COMPAT. For example, day and month short names of GERMAN "de" locale are different in CLDR and COMPAT, just need to add a . in the end for CLDR. Resolves opensearch-project#11550 Signed-off-by: Abdul Muneer Kolarkunnu muneer.kolarkunnu@netapp.com Signed-off-by: akolarkunnu <abdul.kolarkunnu@netapp.com> --------- Signed-off-by: Muneer Kolarkunnu <muneer.kolarkunnu@netapp.com> Signed-off-by: akolarkunnu <abdul.kolarkunnu@netapp.com> Signed-off-by: Abdul Muneer Kolarkunnu muneer.kolarkunnu@netapp.com Co-authored-by: akolarkunnu <abdul.kolarkunnu@netapp.com>
1 parent ac78f0a commit ba0df74

File tree

6 files changed

+13
-19
lines changed

6 files changed

+13
-19
lines changed

CHANGELOG-3.0.md

+1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
1717
### Dependencies
1818

1919
### Changed
20+
- Changed locale provider from COMPAT to CLDR ([13988](https://github.com/opensearch-project/OpenSearch/pull/13988))
2021
- Migrate client transports to Apache HttpClient / Core 5.x ([#4459](https://github.com/opensearch-project/OpenSearch/pull/4459))
2122
- Change http code on create index API with bad input raising NotXContentException from 500 to 400 ([#4773](https://github.com/opensearch-project/OpenSearch/pull/4773))
2223
- Improve summary error message for invalid setting updates ([#4792](https://github.com/opensearch-project/OpenSearch/pull/4792))

buildSrc/src/main/java/org/opensearch/gradle/OpenSearchTestBasePlugin.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ public void execute(Task t) {
110110
if (BuildParams.getRuntimeJavaVersion() == JavaVersion.VERSION_1_8) {
111111
test.systemProperty("java.locale.providers", "SPI,JRE");
112112
} else {
113-
test.systemProperty("java.locale.providers", "SPI,COMPAT");
113+
test.systemProperty("java.locale.providers", "SPI,CLDR");
114114
if (test.getJavaVersion().compareTo(JavaVersion.VERSION_17) < 0) {
115115
test.jvmArgs("--illegal-access=warn");
116116
}

distribution/tools/launchers/src/main/java/org/opensearch/tools/launchers/SystemJvmOptions.java

+1-6
Original file line numberDiff line numberDiff line change
@@ -105,13 +105,8 @@ private static String javaLocaleProviders() {
105105
SPI setting is used to allow loading custom CalendarDataProvider
106106
in jdk8 it has to be loaded from jre/lib/ext,
107107
in jdk9+ it is already within ES project and on a classpath
108-
109-
Due to internationalization enhancements in JDK 9 OpenSearch need to set the provider to COMPAT otherwise time/date
110-
parsing will break in an incompatible way for some date patterns and locales.
111-
//TODO COMPAT will be deprecated in at some point, see please https://bugs.openjdk.java.net/browse/JDK-8232906
112-
See also: documentation in <code>server/org.opensearch.common.time.IsoCalendarDataProvider</code>
113108
*/
114-
return "-Djava.locale.providers=SPI,COMPAT";
109+
return "-Djava.locale.providers=SPI,CLDR";
115110
}
116111

117112
}

gradle/ide.gradle

+1-1
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ if (System.getProperty('idea.active') == 'true') {
8181
}
8282
runConfigurations {
8383
defaults(JUnit) {
84-
vmParameters = '-ea -Djava.locale.providers=SPI,COMPAT'
84+
vmParameters = '-ea -Djava.locale.providers=SPI,CLDR'
8585
if (BuildParams.runtimeJavaVersion > JavaVersion.VERSION_17) {
8686
vmParameters += ' -Djava.security.manager=allow'
8787
}

server/src/internalClusterTest/java/org/opensearch/search/query/SearchQueryIT.java

+8-10
Original file line numberDiff line numberDiff line change
@@ -1914,14 +1914,8 @@ public void testRangeQueryWithTimeZone() throws Exception {
19141914
* Test range with a custom locale, e.g. "de" in this case. Documents here mention the day of week
19151915
* as "Mi" for "Mittwoch (Wednesday" and "Do" for "Donnerstag (Thursday)" and the month in the query
19161916
* as "Dez" for "Dezember (December)".
1917-
* Note: this test currently needs the JVM arg `-Djava.locale.providers=SPI,COMPAT` to be set.
1918-
* When running with gradle this is done implicitly through the BuildPlugin, but when running from
1919-
* an IDE this might need to be set manually in the run configuration. See also CONTRIBUTING.md section
1920-
* on "Configuring IDEs And Running Tests".
19211917
*/
19221918
public void testRangeQueryWithLocaleMapping() throws Exception {
1923-
assert ("SPI,COMPAT".equals(System.getProperty("java.locale.providers"))) : "`-Djava.locale.providers=SPI,COMPAT` needs to be set";
1924-
19251919
assertAcked(
19261920
prepareCreate("test").setMapping(
19271921
jsonBuilder().startObject()
@@ -1938,17 +1932,21 @@ public void testRangeQueryWithLocaleMapping() throws Exception {
19381932

19391933
indexRandom(
19401934
true,
1941-
client().prepareIndex("test").setId("1").setSource("date_field", "Mi, 06 Dez 2000 02:55:00 -0800"),
1942-
client().prepareIndex("test").setId("2").setSource("date_field", "Do, 07 Dez 2000 02:55:00 -0800")
1935+
client().prepareIndex("test").setId("1").setSource("date_field", "Mi., 06 Dez. 2000 02:55:00 -0800"),
1936+
client().prepareIndex("test").setId("2").setSource("date_field", "Do., 07 Dez. 2000 02:55:00 -0800")
19431937
);
19441938

19451939
SearchResponse searchResponse = client().prepareSearch("test")
1946-
.setQuery(QueryBuilders.rangeQuery("date_field").gte("Di, 05 Dez 2000 02:55:00 -0800").lte("Do, 07 Dez 2000 00:00:00 -0800"))
1940+
.setQuery(
1941+
QueryBuilders.rangeQuery("date_field").gte("Di., 05 Dez. 2000 02:55:00 -0800").lte("Do., 07 Dez. 2000 00:00:00 -0800")
1942+
)
19471943
.get();
19481944
assertHitCount(searchResponse, 1L);
19491945

19501946
searchResponse = client().prepareSearch("test")
1951-
.setQuery(QueryBuilders.rangeQuery("date_field").gte("Di, 05 Dez 2000 02:55:00 -0800").lte("Fr, 08 Dez 2000 00:00:00 -0800"))
1947+
.setQuery(
1948+
QueryBuilders.rangeQuery("date_field").gte("Di., 05 Dez. 2000 02:55:00 -0800").lte("Fr., 08 Dez. 2000 00:00:00 -0800")
1949+
)
19521950
.get();
19531951
assertHitCount(searchResponse, 2L);
19541952
}

server/src/test/java/org/opensearch/index/mapper/DateFieldMapperTests.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ public void testChangeLocale() throws IOException {
208208
fieldMapping(b -> b.field("type", "date").field("format", "E, d MMM yyyy HH:mm:ss Z").field("locale", "de"))
209209
);
210210

211-
mapper.parse(source(b -> b.field("field", "Mi, 06 Dez 2000 02:55:00 -0800")));
211+
mapper.parse(source(b -> b.field("field", "Mi., 06 Dez. 2000 02:55:00 -0800")));
212212
}
213213

214214
public void testNullValue() throws IOException {

0 commit comments

Comments
 (0)