Np multimapper param starsolo #1172

nikellepetrillo · 2024-01-10T22:06:58Z

Description

Give your PR a concise yet descriptive title.
Please explain the changes you made here.
Explain the motivation for making this change. What existing problem does the pull request solve?
Mention any issues fixed, addressed, or otherwise related to this pull request, including issue numbers or hard links for issues in other repos.
You can delete these instructions once you have written your PR description.

Checklist

If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP documentation team by tagging either @ekiernan or @kayleemathews in a comment on this PR.

Did you add inputs, outputs, or tasks to a workflow?
Did you modify, delete or move: file paths, file names, input names, output names, or task names?
If you made a changelog update, did you update the pipeline version number?

# Conflicts: # beta-pipelines/skylab/m3c/CondensedSnm3C.wdl

github-actions · 2024-01-10T22:07:09Z

Remember to squash merge!

github-actions · 2024-01-11T13:55:16Z

Remember to squash merge!

…o np_multipper_param_starsolo

github-actions · 2024-01-11T14:04:18Z

Remember to squash merge!

github-actions · 2024-01-11T15:46:17Z

Remember to squash merge!

github-actions · 2024-01-11T16:43:17Z

Remember to squash merge!

ekiernan · 2024-01-11T19:27:44Z

pipelines/skylab/optimus/Optimus.wdl

@@ -237,6 +239,12 @@ workflow Optimus {
    File gene_metrics = GeneMetrics.gene_metrics
    File? cell_calls = RunEmptyDrops.empty_drops_result
    File? aligner_metrics = MergeStarOutputs.cell_reads_out
+    Array[File] soloMultiMappers = STARsoloFastq.matrix_sn_rna


Is this output supposed to be optional?

I dont think so, because we do a touch matrix_sn_rna.mtx in the starsolofastq command, so we will always capture this as an output, even if it is empty

kayleemathews

Added a couple of comments - if you have time to add some descriptions for the new outputs that would be great. Don't worry too much about the wording, though. I can help with that. Thanks!

website/docs/Pipelines/Optimus_Pipeline/README.md

website/docs/Pipelines/Multiome_Pipeline/README.md

…o np_multipper_param_starsolo

github-actions · 2024-01-12T18:35:45Z

Remember to squash merge!

…o np_multipper_param_starsolo

github-actions · 2024-01-12T18:36:33Z

Remember to squash merge!

github-actions · 2024-01-12T18:50:06Z

Remember to squash merge!

github-actions · 2024-01-12T18:51:28Z

Remember to squash merge!

github-actions · 2024-01-12T19:01:40Z

Remember to squash merge!

github-actions · 2024-01-12T19:07:58Z

Remember to squash merge!

github-actions · 2024-01-12T19:09:07Z

Remember to squash merge!

kayleemathews

Thanks for helping out with those docs Nikelle!

kayleemathews · 2024-01-12T19:36:37Z

website/docs/Pipelines/Multiome_Pipeline/README.md

+| multimappers_EM_matrix | UniqueAndMult-EM.mtx | Optional output produced when `soloMultiMappers` is "EM"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|
+| multimappers_Uniform_matrix | UniqueAndMult-Uniform.mtx | Optional output produced when `soloMultiMappers` is "Uniform"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|
+| multimappers_Rescue_matrix | UniqueAndMult-Rescue.mtx | Optional output produced when `soloMultiMappers` is "Rescue"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information. |
+| multimappers_PropUnique_matrix | UniqueAndMult-PropUnique.mtx | Optional output produced when `soloMultiMappers` is "PropUnique"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|
+| cell_barcodes_csv | `<cell_csv>` | Optional output produced when `run_cellbender` is "true"; see CellBender [documentation](https://cellbender.readthedocs.io/en/latest/usage/index.html) and [GitHub repository](https://github.com/broadinstitute/CellBender/tree/master) for more information.|


Suggested change

| multimappers_EM_matrix | UniqueAndMult-EM.mtx | Optional output produced when `soloMultiMappers` is "EM"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| multimappers_Uniform_matrix | UniqueAndMult-Uniform.mtx | Optional output produced when `soloMultiMappers` is "Uniform"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| multimappers_Rescue_matrix | UniqueAndMult-Rescue.mtx | Optional output produced when `soloMultiMappers` is "Rescue"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information. |

| multimappers_PropUnique_matrix | UniqueAndMult-PropUnique.mtx | Optional output produced when `soloMultiMappers` is "PropUnique"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| cell_barcodes_csv | `<cell_csv>` | Optional output produced when `run_cellbender` is "true"; see CellBender [documentation](https://cellbender.readthedocs.io/en/latest/usage/index.html) and [GitHub repository](https://github.com/broadinstitute/CellBender/tree/master) for more information.|

| multimappers_EM_matrix | `UniqueAndMult-EM.mtx` | Optional output produced when `soloMultiMappers` is "EM"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| multimappers_Uniform_matrix | `UniqueAndMult-Uniform.mtx` | Optional output produced when `soloMultiMappers` is "Uniform"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| multimappers_Rescue_matrix | `UniqueAndMult-Rescue.mtx` | Optional output produced when `soloMultiMappers` is "Rescue"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information. |

| multimappers_PropUnique_matrix | `UniqueAndMult-PropUnique.mtx` | Optional output produced when `soloMultiMappers` is "PropUnique"; see STARsolo [documentation](https://github.com/alexdobin/STAR/blob/master/docs/STARsolo.md#multi-gene-reads) for more information.|

| cell_barcodes_csv | `<cell_csv>` | Optional output produced when `run_cellbender` is "true"; see CellBender [documentation](https://cellbender.readthedocs.io/en/latest/usage/index.html) and [GitHub repository](https://github.com/broadinstitute/CellBender/tree/master) for more information.|

These look great. Thanks Nikelle!

website/docs/Pipelines/Optimus_Pipeline/README.md

Co-authored-by: Kaylee Mathews <95316074+kayleemathews@users.noreply.github.com>

github-actions · 2024-01-16T14:14:01Z

Remember to squash merge!

…o np_multipper_param_starsolo

github-actions · 2024-01-16T14:16:47Z

Remember to squash merge!

nikellepetrillo · 2024-01-16T14:26:10Z

runs in terra with the new optional input:
https://app.terra.bio/#workspaces/warp-pipelines/multiome_test_data/job_history/5fed0335-d8ed-4347-9a04-9d98e7a92040

https://app.terra.bio/#workspaces/warp-pipelines/multiome_test_data/job_history/010f6537-f2dd-4f09-bb7c-d6796aabd6d1

ekiernan

Thank you!

* update paths to input files * update jg sample map * Km buildindices docs (#1158) * add buildindices overview doc and diagram * Km rnawithumis and ss2 doc updates (#1157) * update rnawithumis overview * Update rna-with-umis.methods.md * Update rna-with-umis.methods.md * update multi-snSS2 readme * Update multi_snss2.methods.md * Update multi_snss2.methods.md * update multi-snSS2 docs * update SS2 overview doc * fix python script link * Lk pd2448 upstools (#1150) Added paired-tag wrapper and demultiplexing task * PD-2435 Test bwa-mem2 step and run Intel distributed BWA-MEM2 (#1147) * Lk pd2453 add bb tag (#1161) Added option to incorporate BB tag in BAM and use it in SnapATAC2 software. * km paired-tag docs (#1165) * update overview docs - update pipeline version numbers in Multiome and Optimus Overview docs * update multi-SS2 overview doc * Update smart-seq2.methods.md * update multi-SS2 methods doc * Update doc_style.md * add paired-tag overview doc * Update website/docs/Pipelines/PairedTag_Pipeline/README.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Apply suggestions from LK doc review Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> --------- Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Np jprb pd 2353 multiple star rsolo align (#1164) * integrate multiple soloFeatures * updating counting_mode definition * logic * count exons is true * count exons is true * fix the logic * count exons false * count exons false * echos * echos * echos * rearrange logic * rearrange * testing * testing * testing * count exons true * switch counting mode order * try running in scrna * clean up * snrna countexons is true * snrna countexons is true * snrna countexons is false * snrna countexons is true * snrna countexons is false * cleaning up * changelogs * changelogs * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * Update pipelines/skylab/smartseq2_single_nucleus_multisample/MultiSampleSmartSeq2SingleNucleus.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Update pipelines/skylab/multiome/Multiome.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Update pipelines/skylab/optimus/Optimus.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Update pipelines/skylab/paired_tag/PairedTag.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Update pipelines/skylab/slideseq/SlideSeq.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> --------- Co-authored-by: Juan Pablo Ramos Barroso <jpramosbarroso@gmail.com> Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * Np update multiome sci test (#1167) * add summary task * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * Update VerifyTasks.wdl * Update VerifyTasks.wdl * PD-2422 BICAN_Optimus_2nymxis_Oct_2023 (#1152) * Np multimapper param starsolo (#1172) * add summary task * add multimapper option * update optimus plumbing for ease of testing * add echos * add to test * remove some echoes * make mouse snrna json go back to what is in dev * make mouse snrna json go back to what is in dev * add as outputs * typo * changelogs * changelogs * changelogs * update pipeline docs * optional output * optional output * optional output * optional output * docs * docs * docs * Update website/docs/Pipelines/Optimus_Pipeline/README.md Co-authored-by: Kaylee Mathews <95316074+kayleemathews@users.noreply.github.com> * remove optional input to tests --------- Co-authored-by: kayleemathews <kmathews@broadinstitute.org> Co-authored-by: Kaylee Mathews <95316074+kayleemathews@users.noreply.github.com> * added exit code to CompareTabix (#1174) Updated the CompareTabix task in the Verify tasks * Lk pd 2452 add read length check (#1171) * adding read2 length and barcode orientation check task * Lk pd 2464 batch methylome (#1181) Added scatter and preemptibles to snM3C * Np edit resources needed for bwa task / add logic to compareBams (#1183) * add summary task * get a bigger machine * more memory * trying out east1 * go back to central and decrease mem and threads * try different zones * make machine smaller * smaller cpu * smaller cpu * more mem * more mem * 2000 disk * more mem compare bams * more mem compare bams * more mem compare bams * no zones * more mem in comparebams * more mem in comparebams * 725000 * 825000 * 725 * max records 3000000 * max records 3000000 * add logic to fail fast if bams differ in size by 200 mb * Update VerifyTasks.wdl * PD-2483 (#1182) * rc-2483 * update changelog * Update README.md --------- Co-authored-by: kayleemathews <kmathews@broadinstitute.org> Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> * PD-2476: Add task before fastqprocess to find number of splits (#1178) * removed space (#1184) Removed space from Verify Tabix task * Lk pd 2455 pairedtag parsebarcodes (#1186) Added a task to parse cell barcodes from sample barcodes * Np move snm3c from beta pipelines (#1185) * add summary task * move CondensedSnm3C.wdl out of beta * update batch numbers * add sorting to the compare compressed text files * changelogs and versions * revert batch change * batch change * batch change * Km snm3c overview doc (#1179) * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update website/docs/Pipelines/snM3C/README.md Co-authored-by: Nikelle Petrillo <38223776+nikellepetrillo@users.noreply.github.com> * Update README.md * fix num_downstr_bases description * Update README.md --------- Co-authored-by: Nikelle Petrillo <38223776+nikellepetrillo@users.noreply.github.com> * Np fix snm3c test (#1190) * add summary task * batch change in test wdl --------- Co-authored-by: Kaylee Mathews <95316074+kayleemathews@users.noreply.github.com> Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> Co-authored-by: aawdeh <awdeh@broadinstitute.org> Co-authored-by: Nikelle Petrillo <38223776+nikellepetrillo@users.noreply.github.com> Co-authored-by: Juan Pablo Ramos Barroso <jpramosbarroso@gmail.com> Co-authored-by: kayleemathews <kmathews@broadinstitute.org> Co-authored-by: Robert Sidney Cox III <sidcox@broadinstitute.org>

nikellepetrillo added 16 commits November 16, 2023 10:35

add summary task

4d8b21c

Merge remote-tracking branch 'origin/develop' into develop

f0c71c9

# Conflicts: # beta-pipelines/skylab/m3c/CondensedSnm3C.wdl

Merge remote-tracking branch 'origin/develop' into develop

5d6c00d

Merge remote-tracking branch 'origin/develop' into develop

2b65b6e

Merge remote-tracking branch 'origin/develop' into develop

294082c

Merge remote-tracking branch 'origin/develop' into develop

d439840

Merge remote-tracking branch 'origin/develop' into develop

1bb3595

add multimapper option

d503743

update optimus plumbing for ease of testing

693c932

add echos

50b75ac

add to test

10b4248

remove some echoes

b52cbf1

make mouse snrna json go back to what is in dev

1949d4e

make mouse snrna json go back to what is in dev

a330405

add as outputs

e4d2fa2

typo

e75a479

Merge branch 'develop' into np_multipper_param_starsolo

1c6470c

nikellepetrillo added 2 commits January 11, 2024 09:03

changelogs

bab93ad

Merge remote-tracking branch 'origin/np_multipper_param_starsolo' int…

6274b61

…o np_multipper_param_starsolo

nikellepetrillo marked this pull request as ready for review January 11, 2024 15:10

changelogs

3aeca87

changelogs

e8fde9f

ekiernan reviewed Jan 11, 2024

View reviewed changes

update pipeline docs

660c1ec

kayleemathews reviewed Jan 12, 2024

View reviewed changes

website/docs/Pipelines/Optimus_Pipeline/README.md Outdated Show resolved Hide resolved

website/docs/Pipelines/Multiome_Pipeline/README.md Outdated Show resolved Hide resolved

nikellepetrillo and others added 3 commits January 12, 2024 13:32

optional output

fc80cdb

Merge remote-tracking branch 'origin/np_multipper_param_starsolo' int…

bc5f00a

…o np_multipper_param_starsolo

Merge branch 'develop' into np_multipper_param_starsolo

c008ee1

nikellepetrillo added 2 commits January 12, 2024 13:36

optional output

500ff51

Merge remote-tracking branch 'origin/np_multipper_param_starsolo' int…

f56e5da

…o np_multipper_param_starsolo

optional output

e8d9111

optional output

875fa4a

docs

5ce21a4

docs

a8223df

docs

21e8d37

kayleemathews approved these changes Jan 12, 2024

View reviewed changes

Update website/docs/Pipelines/Optimus_Pipeline/README.md

47a6197

Co-authored-by: Kaylee Mathews <95316074+kayleemathews@users.noreply.github.com>

nikellepetrillo added 2 commits January 16, 2024 09:16

remove optional input to tests

e72f185

Merge remote-tracking branch 'origin/np_multipper_param_starsolo' int…

f24bcef

…o np_multipper_param_starsolo

ekiernan approved these changes Jan 16, 2024

View reviewed changes

nikellepetrillo merged commit f3e8d6e into develop Jan 16, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Np multimapper param starsolo #1172

Np multimapper param starsolo #1172

nikellepetrillo commented Jan 10, 2024

github-actions bot commented Jan 10, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

ekiernan Jan 11, 2024 •

edited

Loading

nikellepetrillo Jan 12, 2024

kayleemathews left a comment

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

kayleemathews left a comment

kayleemathews Jan 12, 2024

github-actions bot commented Jan 16, 2024

github-actions bot commented Jan 16, 2024

nikellepetrillo commented Jan 16, 2024

ekiernan left a comment

Np multimapper param starsolo #1172

Np multimapper param starsolo #1172

Conversation

nikellepetrillo commented Jan 10, 2024

Description

Checklist

github-actions bot commented Jan 10, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

github-actions bot commented Jan 11, 2024

ekiernan Jan 11, 2024 • edited Loading

Choose a reason for hiding this comment

nikellepetrillo Jan 12, 2024

Choose a reason for hiding this comment

kayleemathews left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

github-actions bot commented Jan 12, 2024

kayleemathews left a comment

Choose a reason for hiding this comment

kayleemathews Jan 12, 2024

Choose a reason for hiding this comment

github-actions bot commented Jan 16, 2024

github-actions bot commented Jan 16, 2024

nikellepetrillo commented Jan 16, 2024

ekiernan left a comment

Choose a reason for hiding this comment

ekiernan Jan 11, 2024 •

edited

Loading