Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to upload artifacts to SQUAD server #497

Merged
merged 2 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion automated/android/noninteractive-tradefed/tradefed.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ params:
TEST_PATH: "android-cts"
# Specify result format: aggregated or atomic
RESULTS_FORMAT: "aggregated"
# The SQUAD url to be used to upload the result and log files.
# see https://squad.readthedocs.io/en/latest/intro.html#submitting-results.
# SQUAD_ARCHIVE_SUBMIT_TOKEN is used for uploading authentication,
# and must be defined by the submitter as one profile managed token
SQUAD_UPLOAD_URL: ""
# Specify url and token for file uploading.
URL: "https://archive.validation.linaro.org/artifacts/team/qa/"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think URL can go away now that archive.vlo is dead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in case archive.vlo is recovered some time later, the current jobs would still be able to run.
leave it here for compatibility for now.

And I also have a plan to remove this URL and the following TOKEN, and the upload-to-artifactorial.sh script in a separate PR, just one concern is that not sure if there is anyone else uses this test as well, their archive server may still work.

TOKEN: ""
Expand Down Expand Up @@ -65,7 +70,8 @@ run:
- sudo dmesg > ./output/dmesg-host.txt || true
- if ! tar caf tradefed-output-$(date +%Y%m%d%H%M%S).tar.xz ./output; then error_fatal "tradefed - failed to collect results and log files [$ANDROID_SERIAL]"; fi
- ATTACHMENT=$(ls tradefed-output-*.tar.xz)
- ../../utils/upload-to-artifactorial.sh -a "${ATTACHMENT}" -u "${URL}" -t "${TOKEN}"
- if [ -n "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-squad.sh -a "${ATTACHMENT}" -u "${SQUAD_UPLOAD_URL}"; fi
- if [ -z "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-artifactorial.sh -a "${ATTACHMENT}" -u "${URL}" -t "${TOKEN}"; fi
# Send test result to LAVA.
- ../../utils/send-to-lava.sh ./output/result.txt
- userdel testuser -f -r || true
Expand Down
74 changes: 74 additions & 0 deletions automated/utils/upload-to-squad.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#!/bin/sh

ATTACHMENT=""
ARTIFACTORIAL_URL=""
CURL_VERBOSE_FLAG=""
FAILURE_RETURN_VALUE=0

usage() {
echo "Usage: $0 [-a <attachment>] [-u <artifactorial_url>] [-v] [-r]" 1>&2
echo " -a attachment Path to the file to upload" 1>&2
echo " -u squad_url SQUAD_URL where the attachment will be uploaded to" 1>&2
echo " This script will try to fetch the SQUAD_ARCHIVE_SUBMIT_TOKEN" 1>&2
echo " token from (lava_test_dir)/secrets or environments for the upload." 1>&2
echo " -v Pass -v (verbose) flag to curl for debugging." 1>&2
echo " -r Report failure. If the upload fails and this flag is set, the script will exit" 1>&2
echo " with return value 1. If the upload is skipped (no URL or no token found)," 1>&2
echo " this script will still return 0." 1>&2
exit 1
}

while getopts ":a:u:vr" opt; do
case "${opt}" in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"t" is missing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for pointing out! will drop the "t" option here.

a) ATTACHMENT="${OPTARG}" ;;
u) ARTIFACTORIAL_URL="${OPTARG}" ;;
v) CURL_VERBOSE_FLAG="-v" ;;
r) FAILURE_RETURN_VALUE=1 ;;
*) usage ;;
esac
done

if [ -z "${ARTIFACTORIAL_URL}" ]; then
echo "test-attachment skip"
command -v lava-test-case > /dev/null 2>&1 && lava-test-case "test-attachment" --result "skip"
exit 0
fi

if command -v lava-test-reference > /dev/null 2>&1; then
# The 'SQUAD_ARCHIVE_SUBMIT_TOKEN' needs to be defined in 'secrects' dictionary in job
# definition file, or defined in the environment, it will be used.
# One issue here Milosz pointed out:
# If there is lava_test_results_dir set in the job context, "/lava-*" might not be correct.
lava_test_dir="$(find /lava-* -maxdepth 0 -type d | grep -E '^/lava-[0-9]+' 2>/dev/null | sort | tail -1)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a bit problematic. If there is lava_test_results_dir set in the job context, "/lava-*" might not be correct. However I've no idea how to make it work for all cases. I would suggest adding a comment to address this problem for future maintainers.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the comment,

if test -f "${lava_test_dir}/secrets"; then
# shellcheck disable=SC1090
. "${lava_test_dir}/secrets"
fi

if [ -z "${SQUAD_ARCHIVE_SUBMIT_TOKEN}" ]; then
echo "WARNING: SQUAD_ARCHIVE_SUBMIT_TOKEN is empty! File uploading skipped."
echo "test-attachment skip"
command -v lava-test-case > /dev/null 2>&1 && lava-test-case "test-attachment" --result "skip"
exit 0
else
# return is the squad testrun id
return=$(curl ${CURL_VERBOSE_FLAG} --header "Auth-Token: ${SQUAD_ARCHIVE_SUBMIT_TOKEN}" --form "attachment=@${ATTACHMENT}" "${ARTIFACTORIAL_URL}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is SQUAD_ARHIVE_SUBMIT_TOKEN the same thing as ARTIFACTORIAL_TOKEN?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From function level, they could be seen as the same. but they are used for different cases, and will be in different value, so need to be defined as two variables I think.

fi

attachmentBasename="$(basename "${ATTACHMENT}")"
if echo "${return}" | grep -E "^[0-9]+$"; then
# ARTIFACTORIAL_URL will be in the format like this:
# https://qa-reports.linaro.org/api/submit/squad_group/squad_project/squad_build/environment
url_squad=$(echo "${ARTIFACTORIAL_URL}"|sed 's|/api/submit/.*||')
url_uploaded="${url_squad}/api/testruns/${return}/attachments/?filename=${attachmentBasename}"
lava-test-reference "test-attachment" --result "pass" --reference "${url_uploaded}"
else
echo "test-attachment fail"
echo "Expected one SQUAD testrun id returend, but curl returned \"${return}\"."
command -v lava-test-case > /dev/null 2>&1 && lava-test-case "test-attachment" --result "fail"
exit "${FAILURE_RETURN_VALUE}"
fi
else
echo "test-attachment skip"
command -v lava-test-case > /dev/null 2>&1 && lava-test-case "test-attachment" --result "skip"
fi
Loading