Skip to content

Commit 5ed8176

Browse files
dblockdblock
authored
Extract message stashing and unstashing into a class. (opensearch-project#924)
* Extract message stashing and unstashing into a class, add unit tests. Signed-off-by: dblock <dblock@amazon.com> * Added test pipeline for gathering parallel stage messages. Signed-off-by: dblock <dblock@amazon.com> * Added a test script that replicates OpenSearch Jenkinsfile but with echo instead of sh. Signed-off-by: dblock <dblock@amazon.com> * Update developer guide with Groovy code prerequisites. Signed-off-by: dblock <dblock@amazon.com> * Fill out testMessages unit test. Signed-off-by: dblock <dblock@amazon.com> * Simplify gradle defaults. Signed-off-by: dblock <dblock@amazon.com> * Reorganized tests and added placeholders. Signed-off-by: dblock <dblock@amazon.com>
1 parent 734081f commit 5ed8176

23 files changed

+986
-75
lines changed

‎.github/workflows/groovy-tests.yml

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
name: groovy-tests
2+
3+
on: [push, pull_request]
4+
5+
jobs:
6+
test:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- name: Setup Java
10+
uses: actions/setup-java@v1
11+
with:
12+
java-version: 11
13+
- uses: actions/checkout@v2
14+
- name: Run Tests
15+
run: |
16+
./gradlew test --info

‎.github/workflows/tests.yml ‎.github/workflows/python-tests.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: tests
1+
name: python-tests
22

33
on: [push, pull_request]
44

‎.gitignore

+3-1
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ coverage.xml
1212

1313
.gradle/
1414
.settings/
15+
/build/
1516
out.txt
1617

1718
/builds/
1819
/dist/
1920
/test-results/
2021

21-
/.vscode/
22+
/.vscode/
23+

‎.groovylintrc.json

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"extends": "recommended",
3+
"rules": {
4+
"CompileStatic": {
5+
"enabled": false
6+
}
7+
}
8+
}

‎DEVELOPER_GUIDE.md

+15-2
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
- [Pipenv](#pipenv)
77
- [NVM and Node](#nvm-and-node)
88
- [Yarn](#yarn)
9+
- [Java](#java)
910
- [Install Dependencies](#install-dependencies)
1011
- [Run Tests](#run-tests)
1112
- [Build OpenSearch](#build-opensearch)
@@ -76,6 +77,10 @@ nvm install v10.24.1
7677
npm install -g yarn
7778
```
7879

80+
#### Java
81+
82+
This project recommends Java 11 for Jenkins jobs CI. This means you must have a JDK 11 installed with the environment variable `JAVA_HOME` referencing the path to Java home for your JDK installation, e.g. `JAVA_HOME=/usr/lib/jvm/jdk-11`. Download Java 11 from [here](https://adoptium.net/releases.html?variant=openjdk11).
83+
7984
### Install Dependencies
8085

8186
Install dependencies.
@@ -90,13 +95,21 @@ Alternatively, run a command inside the virtualenv with pipenv run.
9095

9196
### Run Tests
9297

93-
This project uses [pytest](https://docs.pytest.org/en/6.x/) to ensure code quality. See [tests](tests).
98+
This project uses [pytest](https://docs.pytest.org/en/6.x/) to ensure Python code quality, and [JUnit](https://junit.org/) for Groovy code. See [tests](tests).
9499

95100
```
96101
$ pipenv run pytest
97102
2 passed in 02s
98103
```
99104

105+
```
106+
$ ./gradlew test
107+
108+
> Task :test
109+
BUILD SUCCESSFUL in 7s
110+
3 actionable tasks: 1 executed, 2 up-to-date
111+
```
112+
100113
### Build OpenSearch
101114

102115
Try running `./build.sh`. It should complete and show usage.
@@ -115,7 +128,7 @@ build.sh: error: the following arguments are required: manifest
115128

116129
### Code Linting
117130

118-
This project uses a [pre-commit hook](https://pre-commit.com/) for linting python code.
131+
This project uses a [pre-commit hook](https://pre-commit.com/) for linting Python code.
119132

120133
```
121134
$ pipenv run pre-commit install

‎build.gradle

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
/*
10+
* This build file was generated by the Gradle 'init' task.
11+
*
12+
* This generated file contains a commented-out sample Java project to get you started.
13+
* For more details take a look at the Java Quickstart chapter in the Gradle
14+
* user guide available at https://docs.gradle.org/3.5/userguide/tutorial_java_projects.html
15+
*/
16+
17+
plugins {
18+
id 'com.mkobit.jenkins.pipelines.shared-library' version '0.10.1'
19+
id 'jacoco'
20+
id 'java'
21+
id 'groovy'
22+
}
23+
24+
repositories {
25+
maven { url 'http://bits.netbeans.org/maven2/' }
26+
maven { url 'https://repo.jenkins-ci.org/releases/' }
27+
jcenter()
28+
maven { url 'https://mvnrepository.com/artifact/' }
29+
mavenLocal()
30+
}
31+
32+
dependencies {
33+
compile group: 'org.assertj', name: 'assertj-core', version: '3.4.1'
34+
compile group: 'com.lesfurets', name:'jenkins-pipeline-unit', version: '1.7'
35+
compile group: 'com.cloudbees', name: 'groovy-cps', version: '1.12'
36+
}
37+
38+
sourceSets {
39+
main {
40+
groovy {
41+
srcDirs = ['src/jenkins']
42+
}
43+
}
44+
45+
test {
46+
groovy {
47+
srcDirs = ['tests/jenkins']
48+
}
49+
}
50+
51+
jobs {
52+
groovy {
53+
srcDirs 'src/jenkins/jobs'
54+
compileClasspath += main.compileClasspath
55+
}
56+
57+
compileClasspath += sourceSets.main.output
58+
runtimeClasspath += sourceSets.main.output
59+
}
60+
}
61+
62+
sharedLibrary {
63+
coreVersion = '2.176.2'
64+
testHarnessVersion = '2.54'
65+
pluginDependencies {
66+
workflowCpsGlobalLibraryPluginVersion = '2.16'
67+
dependency('org.jenkins-ci.plugins', 'pipeline-input-step', '2.8')
68+
}
69+
}

‎gradle.properties

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# The OpenSearch Contributors require contributions made to
5+
# this file be licensed under the Apache-2.0 license or a
6+
# compatible open source license.
7+
#
8+
# Modifications Copyright OpenSearch Contributors. See
9+
# GitHub history for details.
10+
#
11+
12+
org.gradle.warning.mode=none
13+
org.gradle.parallel=true

‎gradle/wrapper/gradle-wrapper.jar

57.8 KB
Binary file not shown.

‎gradle/wrapper/gradle-wrapper.properties

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
#
5+
# The OpenSearch Contributors require contributions made to
6+
# this file be licensed under the Apache-2.0 license or a
7+
# compatible open source license.
8+
#
9+
# Modifications Copyright OpenSearch Contributors. See
10+
# GitHub history for details.
11+
#
12+
13+
distributionBase=GRADLE_USER_HOME
14+
distributionPath=wrapper/dists
15+
distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-all.zip
16+
zipStoreBase=GRADLE_USER_HOME
17+
zipStorePath=wrapper/dists
18+
distributionSha256Sum=11657af6356b7587bfb37287b5992e94a9686d5c8a0a1b60b87b9928a2decde5

‎gradlew

+185
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
#!/usr/bin/env sh
2+
3+
#
4+
# Copyright 2015 the original author or authors.
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
19+
##############################################################################
20+
##
21+
## Gradle start up script for UN*X
22+
##
23+
##############################################################################
24+
25+
# Attempt to set APP_HOME
26+
# Resolve links: $0 may be a link
27+
PRG="$0"
28+
# Need this for relative symlinks.
29+
while [ -h "$PRG" ] ; do
30+
ls=`ls -ld "$PRG"`
31+
link=`expr "$ls" : '.*-> \(.*\)$'`
32+
if expr "$link" : '/.*' > /dev/null; then
33+
PRG="$link"
34+
else
35+
PRG=`dirname "$PRG"`"/$link"
36+
fi
37+
done
38+
SAVED="`pwd`"
39+
cd "`dirname \"$PRG\"`/" >/dev/null
40+
APP_HOME="`pwd -P`"
41+
cd "$SAVED" >/dev/null
42+
43+
APP_NAME="Gradle"
44+
APP_BASE_NAME=`basename "$0"`
45+
46+
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
47+
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
48+
49+
# Use the maximum available, or set MAX_FD != -1 to use that value.
50+
MAX_FD="maximum"
51+
52+
warn () {
53+
echo "$*"
54+
}
55+
56+
die () {
57+
echo
58+
echo "$*"
59+
echo
60+
exit 1
61+
}
62+
63+
# OS specific support (must be 'true' or 'false').
64+
cygwin=false
65+
msys=false
66+
darwin=false
67+
nonstop=false
68+
case "`uname`" in
69+
CYGWIN* )
70+
cygwin=true
71+
;;
72+
Darwin* )
73+
darwin=true
74+
;;
75+
MINGW* )
76+
msys=true
77+
;;
78+
NONSTOP* )
79+
nonstop=true
80+
;;
81+
esac
82+
83+
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
84+
85+
86+
# Determine the Java command to use to start the JVM.
87+
if [ -n "$JAVA_HOME" ] ; then
88+
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
89+
# IBM's JDK on AIX uses strange locations for the executables
90+
JAVACMD="$JAVA_HOME/jre/sh/java"
91+
else
92+
JAVACMD="$JAVA_HOME/bin/java"
93+
fi
94+
if [ ! -x "$JAVACMD" ] ; then
95+
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
96+
97+
Please set the JAVA_HOME variable in your environment to match the
98+
location of your Java installation."
99+
fi
100+
else
101+
JAVACMD="java"
102+
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
103+
104+
Please set the JAVA_HOME variable in your environment to match the
105+
location of your Java installation."
106+
fi
107+
108+
# Increase the maximum file descriptors if we can.
109+
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
110+
MAX_FD_LIMIT=`ulimit -H -n`
111+
if [ $? -eq 0 ] ; then
112+
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
113+
MAX_FD="$MAX_FD_LIMIT"
114+
fi
115+
ulimit -n $MAX_FD
116+
if [ $? -ne 0 ] ; then
117+
warn "Could not set maximum file descriptor limit: $MAX_FD"
118+
fi
119+
else
120+
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
121+
fi
122+
fi
123+
124+
# For Darwin, add options to specify how the application appears in the dock
125+
if $darwin; then
126+
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
127+
fi
128+
129+
# For Cygwin or MSYS, switch paths to Windows format before running java
130+
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
131+
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
132+
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
133+
134+
JAVACMD=`cygpath --unix "$JAVACMD"`
135+
136+
# We build the pattern for arguments to be converted via cygpath
137+
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
138+
SEP=""
139+
for dir in $ROOTDIRSRAW ; do
140+
ROOTDIRS="$ROOTDIRS$SEP$dir"
141+
SEP="|"
142+
done
143+
OURCYGPATTERN="(^($ROOTDIRS))"
144+
# Add a user-defined pattern to the cygpath arguments
145+
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
146+
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
147+
fi
148+
# Now convert the arguments - kludge to limit ourselves to /bin/sh
149+
i=0
150+
for arg in "$@" ; do
151+
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
152+
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
153+
154+
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
155+
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
156+
else
157+
eval `echo args$i`="\"$arg\""
158+
fi
159+
i=`expr $i + 1`
160+
done
161+
case $i in
162+
0) set -- ;;
163+
1) set -- "$args0" ;;
164+
2) set -- "$args0" "$args1" ;;
165+
3) set -- "$args0" "$args1" "$args2" ;;
166+
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
167+
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
168+
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
169+
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
170+
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
171+
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
172+
esac
173+
fi
174+
175+
# Escape application args
176+
save () {
177+
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
178+
echo " "
179+
}
180+
APP_ARGS=`save "$@"`
181+
182+
# Collect all arguments for the java command, following the shell quoting and substitution rules
183+
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
184+
185+
exec "$JAVACMD" "$@"

0 commit comments

Comments
 (0)