Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shellcheck to pre-commit and fix warnings #4954

Open
wants to merge 2 commits into
base: branch-25.04
Choose a base branch
from

Conversation

gforsyth
Copy link
Contributor

@gforsyth gforsyth commented Mar 3, 2025

shellcheck is a fast, static analysis tool for shell scripts. It's good at
flagging up unused variables, unintentional glob expansions, and other potential
execution and security headaches that arise from the wonders of bash (and
other shlangs).

This PR adds a pre-commit hook to run shellcheck on all of the sh-lang
files in the ci/ directory, and the changes requested by shellcheck to make
the existing files pass the check.

xref: rapidsai/build-planning#135

@gforsyth gforsyth requested review from a team as code owners March 3, 2025 19:03
@gforsyth gforsyth requested a review from jameslamb March 3, 2025 19:03
@github-actions github-actions bot added the ci label Mar 3, 2025
@gforsyth gforsyth added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Mar 3, 2025
Copy link
Member

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing this. Left some comments for your consideration.

DGL_CHANNEL="dglteam/label/th23_cu118"
else
CONDA_CUDA_VERSION="12.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love when these shellcheck PRs find unused variables 😁

ci/test_wheel.sh Outdated
@@ -28,5 +28,5 @@ else
--import-mode=append \
--benchmark-disable \
-k "not test_property_graph_mg and not test_bulk_sampler_io" \
./python/${package_name}/${python_package_name}/tests
./python/"${package_name}"/"${python_package_name}"/tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
./python/"${package_name}"/"${python_package_name}"/tests
"./python/${package_name}/${python_package_name}/tests"

Since there isn't anything else here like wildcards that need to be evaluated, I have a mild preference for just wrapping the entire thing in double quotes, instead of having 2 inner sets of double quotes. Think that's a little easier to read and to modify.

Comment on lines 63 to 66
jupyter nbconvert --to script "${NBFILENAME}" --output "${NBTMPDIR}"/"${NBNAME}"-test
echo "${MAGIC_OVERRIDE_CODE}" > "${NBTMPDIR}"/tmpfile
cat "${NBTESTSCRIPT}" >> "${NBTMPDIR}"/tmpfile
mv "${NBTMPDIR}"/tmpfile "${NBTESTSCRIPT}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
jupyter nbconvert --to script "${NBFILENAME}" --output "${NBTMPDIR}"/"${NBNAME}"-test
echo "${MAGIC_OVERRIDE_CODE}" > "${NBTMPDIR}"/tmpfile
cat "${NBTESTSCRIPT}" >> "${NBTMPDIR}"/tmpfile
mv "${NBTMPDIR}"/tmpfile "${NBTESTSCRIPT}"
jupyter nbconvert --to script "${NBFILENAME}" --output "${NBTMPDIR}/${NBNAME}-test"
echo "${MAGIC_OVERRIDE_CODE}" > "${NBTMPDIR}/tmpfile"
cat "${NBTESTSCRIPT}" >> "${NBTMPDIR}/tmpfile"
mv "${NBTMPDIR}/tmpfile" "${NBTESTSCRIPT}"

Similar to my comments above.

hooks:
- id: shellcheck
args: ["--severity=warning"]
files: ^ci/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you expand this a bit? I'd really like to see it cover stuff like https://github.com/rapidsai/cugraph/blob/branch-25.04/datasets/get_test_data.sh and https://github.com/rapidsai/cugraph/blob/branch-25.04/scripts/dask/run-dask-process.sh too

If you want to just remove this exclusion and have it cover all shell scripts in this round, I don't mind that making the PR larger.

That'd also be one less repo to have to modify in the second round of this that covers build.sh scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarks ci conda cuGraph datasets improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants