Using Singularity and Nextflow in the Farm

[ccbaumler]

To use a Singularity container on the Farm simply run module avail | grep "singularity" followed by the version of singularity you would like to use that is available on the farm such as module load spack/singularity/3.8.3.

To create a singularity image (a .sif file), pull from the following repositories:
docker://* A container hosted on Docker Hub
shub://* A container hosted on Singularity Hub
oras://* A container hosted on a supporting OCI registry

$ singularity pull python-3.8.2.sif docker://python:3.8.2-slim-buster

See this for more information.

One extremely useful combination of Farm modules is singularity and nextflow. The below workflow allows

#see if the modules are on the Farm. Put "nextflow" and "singularity" between the quotation marks.
$ module avail | grep "<what-your-looking-for>"

#load these module to use nextflow. Then use nextflow's fetchngs to download sequences.
$ module load spack/singularity/3.8.3

$ module load nextflow/22.10.4

#visualize an accession list.
$ cat ids.csv
SRR9157333
SRR9157334
SRR9157335
SRR9157336
SRR9157337
SRR9157338
SRR9157339
SRR9157340
SRR9157345

#run nextflow's fetchngs with the command below.
$ nextflow run nf-core/fetchngs --input ids.csv --outdir fetchngs -profile singularity -r 1.9 --force_sratools_download

[ccbaumler]

Some suggested materials from Mo on a similar subject:
As a start, I will recommend focusing only on docker.
This video is good to get started with implementation
https://youtu.be/gAkwW2tuIqE
And this playlist is good for learning the concepts
https://youtube.com/playlist?list=PL4cUxeGkcC9hxjeEtdHFNYMtCpjNBm3h7
These will be enough to get started with docker. Once grasped the concepts, you will search for any question very easily

If you want to dive into the details of both docker and Kubernetes, then these crash courses are so good
https://www.youtube.com/watch?v=kTp5xUtcalw&ab_channel=freeCodeCamp.org
Or Kubernetes only: https://www.youtube.com/watch?v=X48VuDVv0do&ab_channel=TechWorldwithNana