Feat: adds private docker container support

[dapineyro]

Overview

This PR adds support for "Use private Docker repository" feature in CloudOS.

This new feature allows the user to run using private docker images from their personal docker.io account.

The users have to link their docker.io account to cloudOS using the UI. If docker.io account is not linked, the user won't see the "Use private Docker repository" option in batch queue options. They will see a link to the page for connecting accounts.

This feature doesn't work with system-tools and similar (Module workflows), as there is no option to select queues and then the option for use private Docker repo is not shown.

Jira

https://lifebit.atlassian.net/browse/LIF-1568

Changes

• Flag --use-private-docker-repository added to cloudos job run command.
• Checks done when using cloudos jog run --use-private-docker-repository:
  1. If the workflow is a system tool or similar (it's a module), then a message will be printed to inform the user the option won't take effect as CloudOS module tools are not compatible
  2. It checks user's credentials and if the user does not have credentials for docker.io, it raises an error
  3. If all passes, the same message as in the UI will be printed to inform the user that the option was selected and a custom job queue will be created based on their selected queue (or the default one)
• The new method Cloudos.get_user_info has been added to be able to inspect user's credentials.
• A new Pytest has been added for the new method.
• The "dockerLogin" API request parameter has been added based on the new parameter.

Tests

Using docker image:

docker run --rm -it quay.io/lifebitaiorg/cloudos-cli:v2.16.0

Global variables:

CLOUDOS="https://cloudos.lifebit.ai"
MY_API_KEY="xxxx"
WORKSPACE_ID="xxxx"
PROJECT_NAME="API jobs"
WORKFLOW_NAME="rnatoy"
JOB_PARAMS="cloudos/examples/rnatoy.config"

Sending a job with default params (not using the new flag)

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name $WORKFLOW_NAME \
    --job-config $JOB_PARAMS \
    --job-name cloudos_cli_default

Sending a job using --use-private-docker-repository from a user that has the credentials linked

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name $WORKFLOW_NAME \
    --job-config $JOB_PARAMS \
    --job-name cloudos_cli_private_docker \
    --use-private-docker-repository

Note

The queue used is now a clone from the previous one

Sending a job using --use-private-docker-repository from a user that doesn't have the credentials linked

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name $WORKFLOW_NAME \
    --job-config $JOB_PARAMS \
    --job-name cloudos_cli_private_docker \
    --use-private-docker-repository

Sending a GWAS job (a module tool) using --use-private-docker-repository from a user that has the credentials linked

WORKFLOW_NAME="GWAS"

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name $WORKFLOW_NAME \
    --nextflow-profile test_ci_staging_cloudos_phenofile \
    --job-name cloudos_cli_system_tool \
    --use-private-docker-repository

Note

It is using the expected system-tools queue

[s0rthak]

LGTM!

[danielboloc]

LGTM!