Skip to content

Commit

Permalink
feat: implement dockerized benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
Pfeil committed Jan 21, 2025
1 parent c1fdc5d commit 0aada09
Show file tree
Hide file tree
Showing 4 changed files with 141 additions and 39 deletions.
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,8 @@ This implies the following properties:
- Building (with tests): `./gradlew clean build`
- Building (with verbose test output) `./gradlew -Dprofile=verbose clean build`
- Building (without tests): `./gradlew clean build -x test`
- Run docker integration tests:
- `./gradlew clean build` (by default, this will reuse the local build)
- `time bash ./docker/test_docker.sh` (runs test script)
- Run docker integration tests: `time bash ./docker/test_docker.sh` (will reuse the local build)
- Run dockerized validation benchmarks: `time bash ./docker/test_docker.sh benchmark` (will reuse the local build)
- Doing a release: `./gradlew clean build release`
- Will prompt you about version number to use and next version number
- Will make a git tag which can later be used in a GitHub release
Expand Down
30 changes: 19 additions & 11 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,33 @@

There are two images:

1. `Dockerfile-build-in-image` will build the image in the container. The result is still a clean image as it uses multi-stage-builds to only include what it really needs to run. This is good for a container release.
2. `Dockerfile-reuse-local-build` will reuse the build on your local machine. This is good for local development and reusing CI artifacts.
- `Dockerfile-build-in-image`
- Will build the image in the container. The result is still a clean image as it uses multi-stage-builds to only include what it really needs to run.
- Best for making a container release.
- Used by CI to create an image for publication.
- `Dockerfile-reuse-local-build`
- Will reuse the build on your local machine.
- Good for local development and reusing CI artifacts.
- Used in `test_docker.sh` and build/test CI.

The CI will use the first Dockerfile to build and provide images.

## Testing
## Run all tests

The test script `docker_tests.sh` will:
For the recommended usage instructions, see the main readme file. The test script `docker_tests.sh` will:

- build and run a container, named `typed-pid-maker-test`
- without parameters, it will use the `Dockerfile-reuse-local-build` definition.
- given an arbitrary argument (we recommend "release" or "build-in-image" for readability), it will use the `Dockerfile-build-in-image` definition.
- execute all tests in the `tests` subfolder
- if the script gets `benchmark` as its first parameter:
- execute benchmark tests repeatedly
- else (default):
- execute all tests in the `tests` subfolder
- stop and delete the container

The idea behind these tests is to have basic tests from a very practical perspective (integration tests, component/service tests). The goal is to test the docker container, but also the standard configuration (application.properties) and the spring setup in general. Examples:
The idea behind the tests is to have basic tests from a very practical perspective (integration tests, component/service tests). The goal is to test the docker container, but also the standard configuration (application.properties) and the spring setup in general. Examples:

- test the creation of a PID: This makes sure that the request actually reaches the handler function. Such a test should exist for all endpoint with different security configurations.
- test the creation of a PID: This makes sure that the request actually reaches the handler function. Such a test should exist for all endpoints with different security configurations.
- test if swagger page is reachable: This makes sure that the spring and openapi/swagger libraries work well together in the current version.
- test if openAPI definition is reachable: same as above.

The goal is **not** to achieve full test coverage here. For this, we have unit tests and integration tests in `src/test`.
The goal is **not** to achieve full test coverage here. For this, we have unit tests and integration tests in `src/test`.

The benchmark mode is available to see the impact of new developments on the validation.
80 changes: 55 additions & 25 deletions docker/test_docker.sh
Original file line number Diff line number Diff line change
@@ -1,46 +1,76 @@
#!/usr/bin/env bash

# hurl is required
if ! command -v hurl &>/dev/null; then
echo "> hurl is required but it's not installed. Aborting." >&2
exit 1
fi

if [ "$1" == "benchmark" ]; then
benchmark_mode=0 # 0 (true) for benchmarks
echo "> benchmark mode"
else
benchmark_mode=1 # 1 (false) for tests
echo "> test mode"
fi

# docker parameters
tag=typed-pid-maker-test
container=typid-test

# meta information for this script
this=${BASH_SOURCE[0]}
echo "this script is at $this"
echo "> this script is at $this"
docker_dir=$(dirname "$this")

echo "build docker image"
echo "> trigger local build"
"$docker_dir"/../gradlew build -x test || exit 1

echo "> build docker image, reusing local build"
sleep .2
# use "standalone" or "release" to build in the docker container
if [ "$1" ]
then
echo " > compiling in container: "
docker build --file $docker_dir/Dockerfile-build-in-image --tag $tag $docker_dir/.. || exit 1
else
echo " > reusing local build: "
docker build --file $docker_dir/Dockerfile-reuse-local-build --tag $tag $docker_dir/.. || exit 1
fi
docker build --file "$docker_dir/Dockerfile-reuse-local-build" \
--tag "$tag" "$docker_dir/.." || exit 1

echo -n "run container: "
docker run -p 8090:8090 --detach --name $container $tag
echo -n "> run container: "
docker run \
--env pit.typeregistry.cache.maxEntries=0 \
-p 8090:8090 \
--detach \
--name $container $tag

echo "> Making sure the service is ready to accept requests"
hurl --retry=20 --retry-interval=5000 \
--test "$docker_dir"/tests/create_resolve_exists.hurl ||
exit 1
echo "> Service is ready. Starting with actual tests."

#####################################
### tests ###########################
### run tests / benchmarks ################
#####################################
hurl --retry=10 --retry-interval=10000 \
--test "$docker_dir"/tests/*.hurl
failure=$?
echo "> benchmark mode: $benchmark_mode"
if [ "$benchmark_mode" -eq 0 ]; then
echo "> running benchmark"
hurl --retry=20 --retry-interval=5000 \
--test --jobs 1 --repeat 1000 \
"$docker_dir"/tests/benchmark-create_dryrun.hurl
failure=$?
else
echo "> running tests"
hurl --retry=20 --retry-interval=5000 \
--test "$docker_dir"/tests/*.hurl
failure=$?
fi
#####################################

echo -n "stopping container ... "
echo -n "> stopping container ... "
docker container stop $container
echo -n "removing container ... "
echo -n "> removing container ... "
docker container rm $container

if [ $failure -eq 0 ]
then
echo "ALL TESTS SUCCESSFUL."
exit 0
if [ $failure -eq 0 ]; then
echo "> ALL TESTS SUCCESSFUL."
exit 0
else
echo "TESTS FAILED (see output for details)"
exit 1
echo "> TESTS FAILED (see output for details)"
exit 1
fi
65 changes: 65 additions & 0 deletions docker/tests/benchmark-create_dryrun.hurl
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# create/register a pid record
POST http://localhost:8090/api/v1/pit/pid/
Content-Type: application/vnd.datamanager.pid.simple+json
Accept: application/vnd.datamanager.pid.simple+json
[QueryStringParams]
dryrun: true
{
"record": [
{
"key": "21.T11148/076759916209e5d62bd5",
"value": "21.T11148/b9b76f887845e32d29f7"
},
{
"key": "21.T11148/1c699a5d1b4ad3ba4956",
"value": "21.T11148/ca9fd0b2414177b79ac2"
},
{
"key": "21.T11148/a753134738da82809fc1",
"value": "21.T11148/a753134738da82809fc1"
},
{
"key": "21.T11148/b8457812905b83046284",
"value": "https://hdl.handle.net/21.T11148/b8457812905b83046284"
},
{
"key": "21.T11148/1a73af9e7ae00182733b",
"value": "https://orcid.org/0000-0001-6575-1022"
},
{
"key": "21.T11148/aafd5fb4c7222e2d950a",
"value": "2020-10-21T00:00:00+02:00"
},
{
"key": "21.T11969/a00985b98dac27bd32f8",
"value": "Book"
},
{
"key": "21.T11148/2f314c8fe5fb6a0063a8",
"value": "{\"licenseURL\": \"https://www.gnu.org/licenses/agpl-3.0.en.html\"}"
},
{
"key": "21.T11148/82e2503c49209e987740",
"value": "{\"md5sum\": \"2289159614f3e3b06fc436423c0dc398\"}"
},
{
"key": "21.T11148/7fdada5846281ef5d461",
"value": "{\"locationPreview/Sample\": \"https://example.com/my/path/to/image.svg\"}"
},
{
"key": "21.T11148/6ae999552a0d2dca14d6",
"value": "this-is-a-string"
},
{
"key": "21.T11148/f3f0cbaa39fa9966b279",
"value": "{\"identifier\": \"this-is-a-string\"}"
},
{
"key": "21.T11148/4fe7cde52629b61e3b82",
"value": "sandboxed/some-random-pid"
}
]
}

# on success, we get a 200 (no 201, because we're in dry-run mode)
HTTP 200

0 comments on commit 0aada09

Please sign in to comment.