Skip to content

Commit 1f8415a

Browse files
authored
Various fixes (opensearch-project#886)
* Various fixes This pull request addresses several issues related to the compiler and tests in the current main branch. The first major change involves replacing ImmutableOpenMap with java.util.Map in the core. This modification is implemented in the following pull requests: opensearch-project/OpenSearch#7165 opensearch-project/OpenSearch#7301 To accommodate this change, the codebase of the AD (Anomaly Detection) module has been refactored to utilize JDK maps. As a consequence of this change, passing null to the custom parameters of ClusterState is no longer permissible, as it leads to a NullPointerException. The error stack trace is as follows: java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "m" is null at __randomizedtesting.SeedInfo.seed([60CDDB34427ACD0C:6E72DB4ED18E018D]:0) at java.base/java.util.Collections.unmodifiableMap(Collections.java:1476) at org.opensearch.cluster.ClusterState.<init>(ClusterState.java:219) at org.opensearch.ad.transport.DeleteAnomalyDetectorTests.createClusterState(DeleteAnomalyDetectorTests.java:216) at org.opensearch.ad.transport.DeleteAnomalyDetectorTests.testDeleteADTransportAction_LatestDetectorLevelTask(DeleteAnomalyDetectorTests.java:160) To address this issue, we have replaced the usage of null with new HashMap<>(). The second change in this pull request aligns with the modifications introduced in PR opensearch-project#878. The third issue is related to the incompatibility between tests that utilize the @parameters annotation and those that do not, as explained in https://tinyurl.com/2y265s2w. Specifically, the SearchFeatureDaoTests class runes tests with the @parameters annotation, whereas SearchFeatureDao tests do not. Testing done: 1. gradle build. Signed-off-by: Kaituo Li <kaituo@amazon.com>
1 parent 0a48da9 commit 1f8415a

16 files changed

+738
-444
lines changed

build.gradle

+11-2
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ buildscript {
3434
js_resource_folder = "src/test/resources/job-scheduler"
3535
common_utils_version = System.getProperty("common_utils.version", opensearch_build)
3636
job_scheduler_version = System.getProperty("job_scheduler.version", opensearch_build)
37-
bwcVersionShort = "2.7.0"
37+
bwcVersionShort = "2.8.0"
3838
bwcVersion = bwcVersionShort + ".0"
3939
bwcOpenSearchADDownload = 'https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/' + bwcVersionShort + '/latest/linux/x64/tar/builds/' +
4040
'opensearch/plugins/opensearch-anomaly-detection-' + bwcVersion + '.zip'
@@ -67,7 +67,8 @@ plugins {
6767
id 'nebula.ospackage' version "8.3.0" apply false
6868
id "com.diffplug.gradle.spotless" version "3.26.1"
6969
id 'java-library'
70-
id 'org.gradle.test-retry' version '1.3.1'
70+
// Gradle 7.6 support was added in test-retry 1.4.0.
71+
id 'org.gradle.test-retry' version '1.4.1'
7172
}
7273

7374
tasks.withType(JavaCompile) {
@@ -77,6 +78,14 @@ tasks.withType(Test) {
7778
systemProperty "file.encoding", "UTF-8"
7879
jvmArgs("--add-opens", "java.base/java.time=ALL-UNNAMED")
7980
jvmArgs("--add-opens", "java.base/java.util.stream=ALL-UNNAMED")
81+
82+
// PowerMock related tests like SearchFeatureDaoTests relies on modifying the bytecode of
83+
// classes during runtime, which can conflict with the module system introduced in Java 9.
84+
// To resolve this issue, we use the --add-opens option to explicitly open the java.util
85+
// and java.lang package to PowerMock. This option allows PowerMock to access non-public
86+
// members of the java.util and java.lang package.
87+
jvmArgs('--add-opens', 'java.base/java.util=ALL-UNNAMED')
88+
jvmArgs('--add-opens', 'java.base/java.lang=ALL-UNNAMED')
8089
}
8190
tasks.withType(Javadoc) {
8291
options.encoding = 'UTF-8'

gradle/wrapper/gradle-wrapper.jar

2.32 KB
Binary file not shown.
+2-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
distributionBase=GRADLE_USER_HOME
22
distributionPath=wrapper/dists
3-
distributionUrl=https\://services.gradle.org/distributions/gradle-7.4.2-bin.zip
3+
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.1-bin.zip
4+
networkTimeout=10000
45
zipStoreBase=GRADLE_USER_HOME
56
zipStorePath=wrapper/dists

gradlew

+164-105
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
#!/usr/bin/env sh
1+
#!/bin/sh
22

33
#
4-
# Copyright 2015 the original author or authors.
4+
# Copyright © 2015-2021 the original authors.
55
#
66
# Licensed under the Apache License, Version 2.0 (the "License");
77
# you may not use this file except in compliance with the License.
@@ -17,67 +17,101 @@
1717
#
1818

1919
##############################################################################
20-
##
21-
## Gradle start up script for UN*X
22-
##
20+
#
21+
# Gradle start up script for POSIX generated by Gradle.
22+
#
23+
# Important for running:
24+
#
25+
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
26+
# noncompliant, but you have some other compliant shell such as ksh or
27+
# bash, then to run this script, type that shell name before the whole
28+
# command line, like:
29+
#
30+
# ksh Gradle
31+
#
32+
# Busybox and similar reduced shells will NOT work, because this script
33+
# requires all of these POSIX shell features:
34+
# * functions;
35+
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
36+
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
37+
# * compound commands having a testable exit status, especially «case»;
38+
# * various built-in commands including «command», «set», and «ulimit».
39+
#
40+
# Important for patching:
41+
#
42+
# (2) This script targets any POSIX shell, so it avoids extensions provided
43+
# by Bash, Ksh, etc; in particular arrays are avoided.
44+
#
45+
# The "traditional" practice of packing multiple parameters into a
46+
# space-separated string is a well documented source of bugs and security
47+
# problems, so this is (mostly) avoided, by progressively accumulating
48+
# options in "$@", and eventually passing that to Java.
49+
#
50+
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
51+
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
52+
# see the in-line comments for details.
53+
#
54+
# There are tweaks for specific operating systems such as AIX, CygWin,
55+
# Darwin, MinGW, and NonStop.
56+
#
57+
# (3) This script is generated from the Groovy template
58+
# https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
59+
# within the Gradle project.
60+
#
61+
# You can find Gradle at https://github.com/gradle/gradle/.
62+
#
2363
##############################################################################
2464

2565
# Attempt to set APP_HOME
66+
2667
# Resolve links: $0 may be a link
27-
PRG="$0"
28-
# Need this for relative symlinks.
29-
while [ -h "$PRG" ] ; do
30-
ls=`ls -ld "$PRG"`
31-
link=`expr "$ls" : '.*-> \(.*\)$'`
32-
if expr "$link" : '/.*' > /dev/null; then
33-
PRG="$link"
34-
else
35-
PRG=`dirname "$PRG"`"/$link"
36-
fi
68+
app_path=$0
69+
70+
# Need this for daisy-chained symlinks.
71+
while
72+
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
73+
[ -h "$app_path" ]
74+
do
75+
ls=$( ls -ld "$app_path" )
76+
link=${ls#*' -> '}
77+
case $link in #(
78+
/*) app_path=$link ;; #(
79+
*) app_path=$APP_HOME$link ;;
80+
esac
3781
done
38-
SAVED="`pwd`"
39-
cd "`dirname \"$PRG\"`/" >/dev/null
40-
APP_HOME="`pwd -P`"
41-
cd "$SAVED" >/dev/null
4282

43-
APP_NAME="Gradle"
44-
APP_BASE_NAME=`basename "$0"`
83+
# This is normally unused
84+
# shellcheck disable=SC2034
85+
APP_BASE_NAME=${0##*/}
86+
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
4587

4688
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
4789
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
4890

4991
# Use the maximum available, or set MAX_FD != -1 to use that value.
50-
MAX_FD="maximum"
92+
MAX_FD=maximum
5193

5294
warn () {
5395
echo "$*"
54-
}
96+
} >&2
5597

5698
die () {
5799
echo
58100
echo "$*"
59101
echo
60102
exit 1
61-
}
103+
} >&2
62104

63105
# OS specific support (must be 'true' or 'false').
64106
cygwin=false
65107
msys=false
66108
darwin=false
67109
nonstop=false
68-
case "`uname`" in
69-
CYGWIN* )
70-
cygwin=true
71-
;;
72-
Darwin* )
73-
darwin=true
74-
;;
75-
MINGW* )
76-
msys=true
77-
;;
78-
NONSTOP* )
79-
nonstop=true
80-
;;
110+
case "$( uname )" in #(
111+
CYGWIN* ) cygwin=true ;; #(
112+
Darwin* ) darwin=true ;; #(
113+
MSYS* | MINGW* ) msys=true ;; #(
114+
NONSTOP* ) nonstop=true ;;
81115
esac
82116

83117
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
@@ -87,9 +121,9 @@ CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
87121
if [ -n "$JAVA_HOME" ] ; then
88122
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
89123
# IBM's JDK on AIX uses strange locations for the executables
90-
JAVACMD="$JAVA_HOME/jre/sh/java"
124+
JAVACMD=$JAVA_HOME/jre/sh/java
91125
else
92-
JAVACMD="$JAVA_HOME/bin/java"
126+
JAVACMD=$JAVA_HOME/bin/java
93127
fi
94128
if [ ! -x "$JAVACMD" ] ; then
95129
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
@@ -98,88 +132,113 @@ Please set the JAVA_HOME variable in your environment to match the
98132
location of your Java installation."
99133
fi
100134
else
101-
JAVACMD="java"
135+
JAVACMD=java
102136
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
103137
104138
Please set the JAVA_HOME variable in your environment to match the
105139
location of your Java installation."
106140
fi
107141

108142
# Increase the maximum file descriptors if we can.
109-
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
110-
MAX_FD_LIMIT=`ulimit -H -n`
111-
if [ $? -eq 0 ] ; then
112-
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
113-
MAX_FD="$MAX_FD_LIMIT"
114-
fi
115-
ulimit -n $MAX_FD
116-
if [ $? -ne 0 ] ; then
117-
warn "Could not set maximum file descriptor limit: $MAX_FD"
118-
fi
119-
else
120-
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
121-
fi
143+
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
144+
case $MAX_FD in #(
145+
max*)
146+
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
147+
# shellcheck disable=SC3045
148+
MAX_FD=$( ulimit -H -n ) ||
149+
warn "Could not query maximum file descriptor limit"
150+
esac
151+
case $MAX_FD in #(
152+
'' | soft) :;; #(
153+
*)
154+
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
155+
# shellcheck disable=SC3045
156+
ulimit -n "$MAX_FD" ||
157+
warn "Could not set maximum file descriptor limit to $MAX_FD"
158+
esac
122159
fi
123160

124-
# For Darwin, add options to specify how the application appears in the dock
125-
if $darwin; then
126-
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
127-
fi
161+
# Collect all arguments for the java command, stacking in reverse order:
162+
# * args from the command line
163+
# * the main class name
164+
# * -classpath
165+
# * -D...appname settings
166+
# * --module-path (only if needed)
167+
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
128168

129169
# For Cygwin or MSYS, switch paths to Windows format before running java
130-
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
131-
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
132-
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
133-
134-
JAVACMD=`cygpath --unix "$JAVACMD"`
135-
136-
# We build the pattern for arguments to be converted via cygpath
137-
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
138-
SEP=""
139-
for dir in $ROOTDIRSRAW ; do
140-
ROOTDIRS="$ROOTDIRS$SEP$dir"
141-
SEP="|"
142-
done
143-
OURCYGPATTERN="(^($ROOTDIRS))"
144-
# Add a user-defined pattern to the cygpath arguments
145-
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
146-
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
147-
fi
170+
if "$cygwin" || "$msys" ; then
171+
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
172+
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
173+
174+
JAVACMD=$( cygpath --unix "$JAVACMD" )
175+
148176
# Now convert the arguments - kludge to limit ourselves to /bin/sh
149-
i=0
150-
for arg in "$@" ; do
151-
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
152-
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
153-
154-
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
155-
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
156-
else
157-
eval `echo args$i`="\"$arg\""
177+
for arg do
178+
if
179+
case $arg in #(
180+
-*) false ;; # don't mess with options #(
181+
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
182+
[ -e "$t" ] ;; #(
183+
*) false ;;
184+
esac
185+
then
186+
arg=$( cygpath --path --ignore --mixed "$arg" )
158187
fi
159-
i=`expr $i + 1`
188+
# Roll the args list around exactly as many times as the number of
189+
# args, so each arg winds up back in the position where it started, but
190+
# possibly modified.
191+
#
192+
# NB: a `for` loop captures its iteration list before it begins, so
193+
# changing the positional parameters here affects neither the number of
194+
# iterations, nor the values presented in `arg`.
195+
shift # remove old arg
196+
set -- "$@" "$arg" # push replacement arg
160197
done
161-
case $i in
162-
0) set -- ;;
163-
1) set -- "$args0" ;;
164-
2) set -- "$args0" "$args1" ;;
165-
3) set -- "$args0" "$args1" "$args2" ;;
166-
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
167-
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
168-
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
169-
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
170-
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
171-
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
172-
esac
173198
fi
174199

175-
# Escape application args
176-
save () {
177-
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
178-
echo " "
179-
}
180-
APP_ARGS=`save "$@"`
200+
# Collect all arguments for the java command;
201+
# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of
202+
# shell script including quotes and variable substitutions, so put them in
203+
# double quotes to make sure that they get re-expanded; and
204+
# * put everything else in single quotes, so that it's not re-expanded.
205+
206+
set -- \
207+
"-Dorg.gradle.appname=$APP_BASE_NAME" \
208+
-classpath "$CLASSPATH" \
209+
org.gradle.wrapper.GradleWrapperMain \
210+
"$@"
211+
212+
# Stop when "xargs" is not available.
213+
if ! command -v xargs >/dev/null 2>&1
214+
then
215+
die "xargs is not available"
216+
fi
217+
218+
# Use "xargs" to parse quoted args.
219+
#
220+
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
221+
#
222+
# In Bash we could simply go:
223+
#
224+
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
225+
# set -- "${ARGS[@]}" "$@"
226+
#
227+
# but POSIX shell has neither arrays nor command substitution, so instead we
228+
# post-process each arg (as a line of input to sed) to backslash-escape any
229+
# character that might be a shell metacharacter, then use eval to reverse
230+
# that process (while maintaining the separation between arguments), and wrap
231+
# the whole thing up as a single "set" statement.
232+
#
233+
# This will of course break if any of these variables contains a newline or
234+
# an unmatched quote.
235+
#
181236

182-
# Collect all arguments for the java command, following the shell quoting and substitution rules
183-
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
237+
eval "set -- $(
238+
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
239+
xargs -n1 |
240+
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
241+
tr '\n' ' '
242+
)" '"$@"'
184243

185244
exec "$JAVACMD" "$@"

0 commit comments

Comments
 (0)