Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix caching problems #83

Merged
merged 8 commits into from
Jan 30, 2024
Merged

Fix caching problems #83

merged 8 commits into from
Jan 30, 2024

Conversation

nictru
Copy link
Collaborator

@nictru nictru commented Nov 19, 2023

This PR fixes some general pipeline problems:

  • Non-escaped tab characters in scripts
  • Mutable changes to the meta dict
  • Problems with caching caused by references to the projectDir (only occur when pipeline output is written to the project dir)

@nictru nictru mentioned this pull request Nov 19, 2023
@nictru nictru self-assigned this Nov 19, 2023
@nictru nictru added the bug Something isn't working label Nov 19, 2023
@nictru nictru changed the title Caching Fix caching problems Nov 19, 2023
@rreggiar rreggiar marked this pull request as ready for review November 27, 2023 16:29
@rreggiar rreggiar self-assigned this Nov 27, 2023
Copy link

@rreggiar rreggiar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleaner implementation of the channel clone than what I had, looks good

@@ -1,3 +1,5 @@
#!/usr/bin/env python

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not for this PR, but in general can you add author and license please to your custom scripts?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script contains the requested information in lines 7-10. Maybe not in the expected location, but the only thing I did here was adding the shebang. Quickly went through all the other scripts, found author and license in various formats in all of them.

Should I unify the formatting of this information across all files?

Comment on lines 11 to 13
tuple val(meta), path(bed, name: "bed.input")
path fasta, name: "fasta.input"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you naming the input here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should better be handled using task.ext.prefix in the modules.config - will quickly do that

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok there was not even a file name collission, don't know why I did that

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I found the reason for this now: There are parts of this process which manipulate all files that end with *.fa and *.fasta. If the input has one of this extensions, it will not work properly. I temporarily re-activated this, since restructuring of the process is not in the scope of this PR but I will handle this soon.

Copy link

github-actions bot commented Jan 10, 2024

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit cf629ae

+| ✅ 162 tests passed       |+
!| ❗   2 tests had warnings |!

❗ Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • system_exit - System.exit in circrna.nf: System.exit(1) [line 43]

✅ Tests passed:

Run details

  • nf-core/tools version 2.11.1
  • Run at 2024-01-28 20:46:45

@nictru nictru linked an issue Jan 13, 2024 that may be closed by this pull request
@nictru nictru force-pushed the caching branch 2 times, most recently from e15a961 to eecedfd Compare January 13, 2024 15:34
Copy link
Contributor

@MariekeVromman MariekeVromman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good as far as I can tell!

@nictru nictru merged commit 9999e23 into dev Jan 30, 2024
6 checks passed
@nictru nictru deleted the caching branch January 30, 2024 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Not all samples are used in analysis resulting in pipeline failur
4 participants