Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container application building #369

Merged
merged 28 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
1f73526
Moved GH workflow reporting into separate job
mluypaert Jan 22, 2024
b6dd309
Moved Dockerfile to repo root and API jar build artefact to dedicated…
mluypaert Jan 22, 2024
68744da
Moved git version retrieval to separate target (to enable target call…
mluypaert Jan 23, 2024
593601c
Split google secret retrieval into independent ENV targets
mluypaert Jan 24, 2024
f241ca0
Enabled (google) secrets storage and sourcing in Makefile
mluypaert Jan 24, 2024
4ea0c50
Enabled building ui without nvm usage (for CI/docker context)
mluypaert Jan 25, 2024
069ff77
Support building app jar to custom path
mluypaert Jan 25, 2024
8a7d6b5
Make build target naming consistent
mluypaert Jan 25, 2024
bb678e9
Build complete application in Dockerfile.
mluypaert Jan 26, 2024
1a0ca06
Enable make run using AWS_PROFILE for AWS authentication (DB access)
mluypaert Jan 26, 2024
1ecec25
Minor cleanup
mluypaert Jan 26, 2024
81ae0cd
Merge branch 'PR-validation' into container-application-building
mluypaert Jan 26, 2024
9037191
Run target naming cleanup
mluypaert Jan 26, 2024
c2fe52a
Added .dockerIgnore to reduce docker image build-time and size
mluypaert Jan 26, 2024
ffee94e
Moved datomic bundle download into separate target (for improved repe…
mluypaert Jan 26, 2024
a7652b4
Prevent accidental secrets leakage
mluypaert Jan 26, 2024
a494521
Added note to document current issues running run-docker using local DB
mluypaert Jan 26, 2024
90e15d9
Added container build and run testing to PR validation
mluypaert Jan 26, 2024
b7d1f79
Fixed editor mishap
mluypaert Jan 26, 2024
70e0451
Allow retrieval of tags for image building in GH actions
mluypaert Jan 26, 2024
d7ccfd7
Prevent errors on cleanup of non-existing containers
mluypaert Jan 26, 2024
8d5eac9
Ensure container run tests cause correct job failure reporting on ste…
mluypaert Jan 26, 2024
4ea84b1
Fix 2 to ensure container accessability tests cause correct job failu…
mluypaert Jan 26, 2024
9522c49
Attempt 3 to fix correct job failure reporting on run-accessability t…
mluypaert Jan 26, 2024
ae20125
Attempt 4 to fix correct job failure reporting on run-accessability t…
mluypaert Jan 26, 2024
cdf92ab
Define default AWS_DEFAULT_REGION value and pass through to container…
mluypaert Jan 26, 2024
af9a7ac
Attempt to pass through temporary AWS credentials retrieved through r…
mluypaert Jan 26, 2024
82b7ca3
Allow more time for application to start up + silenced curl progress …
mluypaert Jan 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
client/node_modules/
client/build/
target/
.github/
.ebextensions/
.elasticbeanstalk/
63 changes: 58 additions & 5 deletions .github/workflows/PR-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ on:
branches:
- master
jobs:
test-api:
permissions:
id-token: write # Required for authentication through OIDC to AWS
reporting:
runs-on: ubuntu-22.04
steps:
- name: Report workflow details
Expand All @@ -20,6 +18,13 @@ jobs:
run: |
git fetch -q origin ${{ github.base_ref }} ${{ github.head_ref }}
git diff --name-only origin/${{ github.base_ref }} origin/${{ github.head_ref }}
test-api:
permissions:
id-token: write # Required for authentication through OIDC to AWS
runs-on: ubuntu-22.04
steps:
- name: Check out repository code
uses: actions/checkout@v3
- name: Set up JDK 8
uses: actions/setup-java@v3
with:
Expand Down Expand Up @@ -50,5 +55,53 @@ jobs:
clojure -Spom
- name: Run Integration tests
run: |
make run-tests GOOGLE_APP_PROFILE=dev
#TODO: add UI and API build and container packaging test
make run-tests APP_PROFILE=dev
build-and-run-container:
permissions:
id-token: write # Required for authentication through OIDC to AWS
runs-on: ubuntu-22.04
steps:
- name: Check out repository code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: AWS credentials configuration
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{secrets.GH_ACTIONS_AWS_ROLE}}
role-session-name: gh-actions-${{github.run_id}}.${{github.run_number}}.${{github.run_attempt}}-test-api
aws-region: us-east-1
mask-aws-account-id: true
- name: Build container image
run: |
make build-docker-image
- name: Launch test instance of container (connecting to test env DB)
run: |
make run-docker PROJ_NAME=wormbase-names-test
sleep 120
- name: Test API accessability
id: test-api-accessability
continue-on-error: true
run: |
curl --no-progress-meter -I http://localhost:3000/api/auth/identity
- name: Test UI accessability
id: test-ui-accessability
continue-on-error: true
run: |
curl --no-progress-meter -I http://localhost:3000/
- name: Report container logs if either accessibility test fails
if: ${{ steps.test-ui-accessability.outcome == 'failure' || steps.test-api-accessability.outcome == 'failure' }}
run: |
docker logs wormbase-names-test
- name: Report UI accessibility test failures
if: ${{ steps.test-ui-accessability.outcome == 'failure' }}
run: |
echo "UI accessability test step failed."
- name: Report API accessibility test failures
if: ${{ steps.test-api-accessability.outcome == 'failure' }}
run: |
echo "API accessability test step failed."
- name: Fail if either accessibility test fails
if: ${{ steps.test-ui-accessability.outcome == 'failure' || steps.test-api-accessability.outcome == 'failure' }}
run: |
exit 1
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ target/
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml
Dockerrun.aws.json
/docker/app.jar
/.dockerignore
build/*

#Local docker-compose env-variable files
.env

# secrets
/resources/secrets/
secrets.makedef

# Generated assets
/resources/client_build
Expand Down
55 changes: 55 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
ARG WORKDIR=/wb_names
ARG APPJAR_BUILDPATH=build/wb-names-app.jar

### Stage 1: build UI
FROM node:12 AS BUILD_UI_STAGE

ARG WORKDIR

WORKDIR $WORKDIR
COPY Makefile ./
COPY client/ ./client/
COPY scripts/ ./scripts/

RUN --mount=type=secret,id=make-secrets-file,required=true \
make build-ui SECRETS_SRC=/run/secrets/make-secrets-file APP_PROFILE=prod

### Stage 2: build API (and include UI components)
FROM clojure:temurin-8-tools-deps-jammy as BUILD_API_STAGE

ARG WORKDIR
ARG APPJAR_BUILDPATH

RUN apt update && apt upgrade -y && apt install -y maven unzip

#Install clojure manually (com.datomic/datomic-pro 1.0.6165 is not stored on maven central)
COPY build/datomic-pro-1.0.6165.zip datomic-pro-1.0.6165.zip
RUN unzip datomic-pro-1.0.6165.zip \
&& cd datomic-pro-1.0.6165/ \
&& bin/maven-install

WORKDIR $WORKDIR
COPY Makefile ./
COPY src/wormbase/ src/wormbase/
COPY resources/ ./resources/
COPY deps.edn ./
COPY project.clj ./
COPY scripts/ ./scripts/
COPY --from=BUILD_UI_STAGE $WORKDIR/client/ $WORKDIR/client/

RUN make build-app-jar APP_JAR_PATH=$APPJAR_BUILDPATH

### Stage 3: build final application image
FROM openjdk:8-jre-alpine as APPLICATION_IMAGE_STAGE

ARG WORKDIR
ARG APPJAR_BUILDPATH

RUN apk update && apk upgrade

COPY --from=BUILD_API_STAGE $WORKDIR/$APPJAR_BUILDPATH /srv/wb-names-app.jar

# Expose necessary ports
EXPOSE 3000

CMD ["java", "-cp", "/srv/wb-names-app.jar", "clojure.main", "-m", "wormbase.names.service"]
Loading
Loading