Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Test all Python versions on Windows on scheduled runs #13191

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ichard26
Copy link
Member

@ichard26 ichard26 commented Jan 29, 2025

Cleaner version of #13014. See #13016 for why we decided to only run Windows CI across all Python versions on a scheduled (currently weekly) basis over all of the time.

Closes #13011.

@ichard26 ichard26 force-pushed the scheduled-windows-ci branch from 1499446 to 2bb3ed9 Compare January 29, 2025 17:39
@ichard26 ichard26 added the skip news Does not need a NEWS file entry (eg: trivial changes) label Jan 29, 2025
@ichard26 ichard26 force-pushed the scheduled-windows-ci branch 2 times, most recently from eaabffe to 5f85788 Compare January 29, 2025 17:43
@ichard26 ichard26 changed the title wip ci: Test all Python versions on Windows on scheduled runs Jan 29, 2025
@ichard26
Copy link
Member Author

And now all the Windows jobs are being disabled. Great. I'll take a look later.

We should test pip across our entire Python support matrix on Windows,
but we don't want to run the test suite across every Python version
on very run as it's slow and unlikely to undercover a bug that the
boundary jobs won't.
@ichard26 ichard26 force-pushed the scheduled-windows-ci branch from 5f85788 to 02277f4 Compare February 12, 2025 01:24
@ichard26
Copy link
Member Author

ichard26 commented Feb 12, 2025

OK, this is somewhat convoluted, but it does allow us to avoid duplicating the entire Windows job definition. I pulled the idea from https://github.com/orgs/community/discussions/26253#discussioncomment-3250989.

Alternatively, if this is too convoluted, we could generate the Windows matrix dynamically in a preceding job and use that to configure the Windows jobs, like https://github.com/FFY00/arch-python-repo/blob/main/.github/workflows/build.yml.

@ichard26 ichard26 marked this pull request as ready for review February 12, 2025 01:26
@ichard26
Copy link
Member Author

I tested the syntax in a separate repository:

@ichard26
Copy link
Member Author

@sbidoul how do you do feel about this approach?

Copy link
Member

@sbidoul sbidoul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Do you know who will be notified of failures of the scheduled jobs?

@ichard26
Copy link
Member Author

I remember being sent a GitHub notification when a workflow failed on a scheduled run, but let's consult the GHA documentation:

Notifications for scheduled workflows are sent to the user who initially created the workflow. If a different user updates the cron syntax in the workflow file, subsequent notifications will be sent to that user instead. If a scheduled workflow is disabled and then re-enabled, notifications will be sent to the user who re-enabled the workflow rather than the user who last modified the cron syntax.

Wow, this is not great. It's probably best to create an issue if the scheduled run fails... I'd rather not use a third-party action for this, so I'll try using the gh tool.

@ichard26 ichard26 added the state: blocked Can not be done until something else is done label Feb 23, 2025
@ichard26
Copy link
Member Author

Also, CI is still too flaky (due to #13153) to open a new issue every time a scheduled run fails. This means this is blocked until that issue is fixed.

@ichard26 ichard26 marked this pull request as draft February 23, 2025 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news Does not need a NEWS file entry (eg: trivial changes) state: blocked Can not be done until something else is done
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pip releases should test against all versions of Python for Windows
2 participants