Add support to upload artifacts to SQUAD server #497

liuyq · 2024-02-26T11:32:37Z

With the automated/utils/upload-to-squad.sh added and the change on the noninteractive-tradefe test as an example in this pull request. there are other tests need to be update as well after pull request.

The SQUAD url should be defined in the format like described here:
https://squad.readthedocs.io/en/latest/intro.html#submitting-results

And the job definition, SQUAD_ARCHIVE_SUBMIT_TOKEN should be defined under the secrets section.

secrets:
  SQUAD_ARCHIVE_SUBMIT_TOKEN: SQUAD_ARCHIVE_SUBMIT_TOKEN

And SQUAD_UPLOAD_URL should be defined as one parameter like:

- test:
    ...
    definitions:
    - repository: https://github.com/Linaro/test-definitions.git
      from: git
      path: automated/android/noninteractive-tradefed/tradefed.yaml
      params:
        ...
        SQUAD_UPLOAD_URL: https://qa-reports.linaro.org/api/submit/squad_group/squad_project/squad_build/environment 
      name: cts-lkft

bhcopeland · 2024-02-26T11:35:32Z

LGTM

mwasilew · 2024-02-26T11:41:22Z

automated/utils/upload-to-squad.sh

+}
+
+while getopts ":a:u:t:vr" opt; do
+    case "${opt}" in


"t" is missing

thanks for pointing out! will drop the "t" option here.

mwasilew · 2024-02-26T11:41:55Z

automated/utils/upload-to-squad.sh

+FAILURE_RETURN_VALUE=0
+
+usage() {
+    echo "Usage: $0 [-a <attachment>] [-u <artifactorial_url>] [-t <artifactorial_token>] [-v] [-r]" 1>&2


"t" option is missing from the help text.

thanks for pointing out it! will drop the "t" description from here.
The idea here is to not support passing the token to avoid the possibility of leak.

mwasilew · 2024-02-26T11:43:47Z

automated/utils/upload-to-squad.sh

+fi
+
+if command -v lava-test-reference > /dev/null 2>&1; then
+    # The 'ARTIFACTORIAL_TOKEN' needs to be defined in 'secrects' dictionary in job


this part contradicts usage description.

will update

mwasilew · 2024-02-26T11:45:27Z

automated/utils/upload-to-squad.sh

+        exit 0
+    else
+        # return is the squad testrun id
+        return=$(curl ${CURL_VERBOSE_FLAG} --header "Auth-Token: ${SQUAD_ARCHIVE_SUBMIT_TOKEN}" --form "attachment=@${ATTACHMENT}" "${ARTIFACTORIAL_URL}")


is SQUAD_ARHIVE_SUBMIT_TOKEN the same thing as ARTIFACTORIAL_TOKEN?

From function level, they could be seen as the same. but they are used for different cases, and will be in different value, so need to be defined as two variables I think.

mwasilew · 2024-02-26T11:46:53Z

automated/utils/upload-to-squad.sh

+
+    attachmentBasename="$(basename "${ATTACHMENT}")"
+    if echo "${return}" | grep -E "^[0-9]+$"; then
+        url_uploaded="https://qa-reports.linaro.org/api/testruns/${return}/attachments/?filename=${attachmentBasename}"


shouldn't ARTIFACTORIAL_URL be used instead of hardcoded qa-reports.linaro.org?

thanks for the suggestion, will update to use ARTIFACTORIAL_URL to get the new url.

liuyq · 2024-02-26T12:55:28Z

@mwasilew just updated according to your comments/suggestions, please help check it again

mwasilew

LGTM with one minor comment. Thanks for fixing it!

mwasilew · 2024-02-26T13:14:53Z

automated/utils/upload-to-squad.sh

+if command -v lava-test-reference > /dev/null 2>&1; then
+    # The 'SQUAD_ARCHIVE_SUBMIT_TOKEN' needs to be defined in 'secrects' dictionary in job
+    # definition file, or defined in the environment, it will be used.
+    lava_test_dir="$(find /lava-* -maxdepth 0 -type d | grep -E '^/lava-[0-9]+' 2>/dev/null | sort | tail -1)"


this is a bit problematic. If there is lava_test_results_dir set in the job context, "/lava-*" might not be correct. However I've no idea how to make it work for all cases. I would suggest adding a comment to address this problem for future maintainers.

updated the comment,

to replace the uploading to archive.vlo Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>

via specifying the SQUAD_UPLOAD_URL variables. To be noted, to avoid leak of tokens, the SQUAD_ARCHIVE_SUBMIT_TOKEN used to upload is not supported to be defined in the job definition in the plain text format, it's must be defined as one profile managed token by the submitter Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>

chaws · 2024-02-26T13:38:29Z

automated/android/noninteractive-tradefed/tradefed.yaml

+    # see https://squad.readthedocs.io/en/latest/intro.html#submitting-results.
+    # SQUAD_ARCHIVE_SUBMIT_TOKEN is used for uploading authentication,
+    # and must be defined by the submitter as one profile managed token
+    SQUAD_UPLOAD_URL: ""
    # Specify url and token for file uploading.
    URL: "https://archive.validation.linaro.org/artifacts/team/qa/"


I think URL can go away now that archive.vlo is dead?

in case archive.vlo is recovered some time later, the current jobs would still be able to run.
leave it here for compatibility for now.

And I also have a plan to remove this URL and the following TOKEN, and the upload-to-artifactorial.sh script in a separate PR, just one concern is that not sure if there is anyone else uses this test as well, their archive server may still work.

liuyq · 2024-02-26T16:09:40Z

Hi, @bhcopeland @roxell @sumitsemwal
could we go ahead to have this one and the #498 one merged first now?

liuyq requested review from chaws, bhcopeland, roxell and vishalbhoj February 26, 2024 11:34

mwasilew suggested changes Feb 26, 2024

View reviewed changes

liuyq requested a review from sumitsemwal February 26, 2024 12:08

liuyq force-pushed the master branch from f5bb40b to cb2354a Compare February 26, 2024 12:34

mwasilew approved these changes Feb 26, 2024

View reviewed changes

liuyq added 2 commits February 26, 2024 21:30

upload-to-squad.sh: add support to upload archives to SQUAD

17783b3

to replace the uploading to archive.vlo Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>

liuyq force-pushed the master branch from cb2354a to df99980 Compare February 26, 2024 13:31

chaws reviewed Feb 26, 2024

View reviewed changes

liuyq mentioned this pull request Feb 26, 2024

Add uploading support for the android apk-automation and boottime tests #498

Merged

bhcopeland approved these changes Feb 26, 2024

View reviewed changes

bhcopeland merged commit 178f445 into Linaro:master Feb 26, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support to upload artifacts to SQUAD server #497

Add support to upload artifacts to SQUAD server #497

liuyq commented Feb 26, 2024

bhcopeland commented Feb 26, 2024 •

edited

Loading

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

liuyq commented Feb 26, 2024

mwasilew left a comment

mwasilew Feb 26, 2024

liuyq Feb 26, 2024

chaws Feb 26, 2024

liuyq Feb 26, 2024

liuyq commented Feb 26, 2024

Add support to upload artifacts to SQUAD server #497

Add support to upload artifacts to SQUAD server #497

Conversation

liuyq commented Feb 26, 2024

bhcopeland commented Feb 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuyq commented Feb 26, 2024

mwasilew left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuyq commented Feb 26, 2024

bhcopeland commented Feb 26, 2024 •

edited

Loading