Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev store downsample data #94

Merged
merged 215 commits into from
Nov 15, 2024
Merged

Dev store downsample data #94

merged 215 commits into from
Nov 15, 2024

Conversation

chrisgulvik
Copy link
Contributor

@chrisgulvik chrisgulvik commented Aug 28, 2024 •
edited
Loading

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <outdir>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@chrisgulvik
save more stats, logs, and intermediate FastQ files prior to final cl…
5cfad96 
…eaned reads
@chrisgulvik
save subsampled FastQ only for seqtk; save cleaned FastQ for hostile
29c1cab
@chrisgulvik
hostile files stored without dirname and .clean_[12] name that hostil…
c48f3f5 
…e adds to it
@chrisgulvik
add process_medium to RDP Classifier to avoid wasted first run fail d…
4fcc82b 
…ue to not enough memory (RAM)
@chrisgulvik
store intermediate Fastp output FastQ files as well
23716f9
@chrisgulvik
migrated from greg to staph-b for container -- staph-b now has v1.4 w…
3c4fc9c 
…ith tests
@chrisgulvik
migrate to v1.4 docker container now that staph-b has it with tests
458fa35
@chrisgulvik
rename Fastp summary filename
98a5d1d
@chrisgulvik
save system call stdout stderr logfiles for all of the missing modules
c19a44c
@chrisgulvik
add default seed to downsampling and hide new seed parameters by default
dcb28f9
@chrisgulvik
add parameter option for subsample_tool to include seqkit as well as …
15667d8 
…default seqtk
@chrisgulvik
edit summary TSV handling, which now creates XLSX
b22d6d3
@chrisgulvik
use collectFile() for CheckM2 TSV Summary file; commented out previou…
26228bc 
…s one-liner that tried to do this
@chrisgulvik
add SeqKit as subsampling option; add toLower() function
b1abf4c
@chrisgulvik
fix shell comparison if ${pplacer_scratch} path is set and not empty
3516f1a
@chrisgulvik
add params.config with default seed value for seqkit
cf1a58c
@chrisgulvik
add new module for subsampling FastQ reads with SeqKit
ff4aef9
@chrisgulvik
set default seed value for seqtk
28a0246
@chrisgulvik
create default params.seqtk_seed set to 947266746; print with msg the…
731ed07 
… seed value used for R1 and R2; cleanup comments
@chrisgulvik
add --create_excel_outputs to the UGE runner script
2e4e741
@chrisgulvik
use locally stored CheckM2 database in the runner script
afff474
@chrisgulvik
fixed missing brackets
116914c 
fixed by turning on Rainbow Brackets (now built-in to VSCode) but had to switch "Editor › Guides: Bracket Pairs" to "true" in "@id:editor.bracketPairColorization.enabled @id:editor.guides.bracketPairs" settings search inside VSCode; `prettier -w .` and `nf-core lint --fix` both did not fix this issue
@chrisgulvik
revise blastn output file to use underscores instead of spaces in hea…
c9f77c9 
…der column names
@chrisgulvik
revise Quast TSV output file to use underscores instead of spaces in …
122b547 
…header line only
@chrisgulvik
revise CheckM2.report.tsv output file to use underscores instead of s…
05f2d13 
…paces in the header line
@chrisgulvik
bugfix: change spaces to underscores in the CheckM2.report.tsv
55db5e0
@chrisgulvik
replace spaces with underscores in header of .FLASH.tsv files
025009a
@chrisgulvik
replace spaces with lines in TSV header output of AlignmentStats
0f64a7f
@chrisgulvik
add feature that reports stdev beside the mean depth of coverage valu…
7f0262d 
…es (sum of squares math in awk!)
@chrisgulvik
replaced spaces with underscored in header of CleanedReads-Bases.tsv …
d766392 
…output file
chrisgulvik and others added 16 commits October 7, 2024 11:55
@chrisgulvik
store novel FastA when that situation occurs
671b5cd
@chrisgulvik
discard extra args related to gene finding
d2c62c4
@chrisgulvik
added wf version to XLSX outfile
eaf4b16
@chrisgulvik
increase process to high due to bottleneck
0a07353
@chrisgulvik
added colors for nondefault processes such as CheckM2, BUSCO, and GTD…
07a1817 
…B-Tk
@chrisgulvik
added output path and additional debug messages throughout
3929aa2
@chrisgulvik
revised bash comparison due to nf-core bug
23925da
@chrisgulvik
added metrics and checksums to each host remove step
93c1cec
@chrisgulvik
revised output statement when <= 0x depth given
bc69051
@chrisgulvik
revised QC Trim output filename; added wf version number to XLSX summ…
342f88d 
…ary outfile
@chrisgulvik
revise input requirements; changed BUSCO, CheckM2, and GTDB-Tk processes
2a75dc6
@chrisgulvik
revise input requirements; changed BUSCO, CheckM2, and GTDB-Tk processes
1191201
@chrisgulvik
changed mash_db default because GTDB-Tk doesnt use external, only loc…
f3aae9b 
…ally built one so it isnt necessary
@chrisgulvik
shortened withName: statements; revised output dir structure for Asse…
0cfe523 
…mbly/QA and Taxonomy/
@chrisgulvik
changed date format for output files; added additional bash variable …
43a48d0 
…quoting
@github-actions
[MegaLinter] Apply linters fixes
c793b2e
@chrisgulvik chrisgulvik force-pushed the dev-store-downsample-data branch from 159501d to c223b7c Compare November 15, 2024 20:29
@chrisgulvik chrisgulvik merged commit 6231bc5 into dev Nov 15, 2024
1 check passed
@chrisgulvik chrisgulvik mentioned this pull request Nov 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@taylorpaisie taylorpaisie taylorpaisie approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@chrisgulvik @taylorpaisie