Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add nix flake workflows #360

Merged
merged 2 commits into from
Feb 10, 2025
Merged

Conversation

MattSturgeon
Copy link
Contributor

@MattSturgeon MattSturgeon commented Jan 31, 2025

Build nix flake on push

Added a workflow that checks the flake and builds the nix package on push.

It may be nice to add a public binary cache, as this could be used as a substituter both by CI and end-users. See below.

Update flake.lock weekly

Added a workflow that schedules a nix flake.lock update every Saturday at noon. This will open a PR with the bumped lockfile.

As-is, CI will not run against the created update PRs, because pushes by "bots" do not trigger CI. There are workarounds available, but they are kinda ugly.

The cleanest solution may be to create a "machine user" account and pass its Personal Access Token to the update-flake-lock action.

See Running GitHub Actions CI and this quote from GitHub's docs regarding "machine users":

User accounts are intended for humans, but you can create accounts to automate activity on GitHub. This type of account is called a machine user. For example, you can create a machine user account to automate continuous integration (CI) workflows.

Binary Cache

With your other build workflows, you often upload the binary artifacts to github actions. For nix, this doesn't really make sense. However it does make sense to upload binaries to a "substitutor" like cachix or flakehub cache.

Such substitutors can be used by both CI, devs, and end-users to reduce re-builds and instead download pre-built binaries.

I haven't included anything related to this in this PR, but if you setup the appropriate accounts & secrets I can do so either here or in a follow-up.

I would probably recommend cachix over flakehub cache. Both require some setup by end-users, but flakehub cache requires end-users install a determinate package and be authenticated with flakehub in order to benefit from caching.

To setup cachix, first create an account. This can be a free account; the free plan grants 5GB of public cache storage, which should be plenty for umu.

Next, as per the Getting Started docs, you'll need to create a cache. This cache should be named something like open-wine-components or umu-launcher, depending on whether you'd use this cache or another for other OWC repos.

Also as per the Getting Started docs, create an auth token for the new cache. Per-cache tokens allow access to only a specific binary cache, rather than all caches in your account. On dashboard you can click on your cache's Settings and generate a new access token. You should give this token a description like "Github CI token", permissions read+write, expires never.

If you named the cache umu-launcher, then your cache's auth settings are here: https://app.cachix.org/cache/umu-launcher/settings/authtokens

Once generated, save the token to GitHub secrets. Name it something like UMU_CACHIX_TOKEN

Use to org secrets if you may end up using the same cache+token in other OWC repos. Org secrets can be added here.
Or use repo secrets if you'd probably create separate caches for other repos in the org. Repo secrets can be added here.

Once this is all set-up, we'd be able to use the cachix action in the workflow:

- name: Setup cachix
  uses: cachix/cachix-action@v15
  with:
    # name of the cache
    name: umu-launcher
    # per-cache auth token
    authToken: ${{ secrets.UMU_CACHIX_TOKEN }}

We may also want to add instructions to the README, for this we'd need to know the cache's name and its "public key":

> [!TIP]
> You can use our binary cache to reduce rebuilds:
>
> ```nix
> nix.settings = {
>   extra-substituters = [
>     "https://umu-launcher.cachix.org"
>   ];
>   extra-trusted-public-keys = [
>     "umu-launcher.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
>   ];
> };
> ```

As ever with github workflows, this is hard to test. So I may have missed some obvious errors.

If I get chance to test these workflows out on a fork (or something), I'll update the PR to clarify that I've done so.

EDIT: Seems the check&build workflow was triggered and ran fine against this PR. The update workflow can only really be tested on a fork or once merged.


cc @beh-10257 @LovingMelody @R1kaB3rN

@LovingMelody
Copy link
Contributor

It currently uses "magic-nix-cache-action", which is based on github-actions' built-in cache.

This action is deprecated and no longer works after today. It's instead adviced to use flakehub cache or cachix

@MattSturgeon
Copy link
Contributor Author

It currently uses "magic-nix-cache-action", which is based on github-actions' built-in cache.

This action is deprecated and no longer works after today. It's instead adviced to use flakehub cache or cachix

Agreed, either cachix or flakehub would be better.

I wasn't aware that the magic caching API is now end of life, so thanks for making me aware of that!

My original intention was to use a simple solution with no additional setup/secrets required in this PR, and then work with the umu repo/org admins in a follow-up PR dedicated to cachix or flakehub. That way each PR can be small and focused.

Given the magic caching API is now EOL, I'll drop it from this PR. Unfortunately, that means the only caching will be the NixOS binary substituters. But the umu build isn't too intensive, so this shouldn't be a problem short-term.

@GloriousEggroll
Copy link
Member

@MattSturgeon is this ready? We're about to get 1.2.0 before SCaLE, would be nice to have

@MattSturgeon
Copy link
Contributor Author

@MattSturgeon is this ready?

Yes, it should be ready.

The build workflow has been working when running against this PR.

The auto update workflow can't be tested until merged, but it is fairly simple and similar to workflows used elsewhere.


There are two non-critical improvements mentioned in the PR description: A "machine user" for the update workflow, and a "binary cache" for the build workflow.

I can't do either on my own; I'd need to work with an org/repo owner to set up accounts & secrets. But as both are "nice to haves" we can merge this PR and then discuss those later.

@R1kaB3rN
Copy link
Member

The workflows provided in this pull request suffices. What's most important is that we have a way to verify that the NixOS package isn't broken for any new changes from contributors/maintainers. For the binary cache and machine user workflows, I would like to see strong use cases to have them from more Nix users.

@R1kaB3rN
Copy link
Member

Linking relevant upstream issue dependabot/dependabot-core#7340 because it would be nice to have this natively in dependabot core. Besides dependabot, it looks automating dependencies via renovate may be possible.

If you can confirm that, I'd be open to transition to it instead.

@LovingMelody
Copy link
Contributor

Unless we have binaries that actually takes a long time or a long-running test suite, I don't think there's really a purpose to add it.

@MattSturgeon
Copy link
Contributor Author

Unless we have binaries that actually takes a long time or a long-running test suite, I don't think there's really a purpose to add it.

The main reason to have binary cache here would be to save on your monthly CI minutes. If you aren't concerned with the current workflow's run-time, then having a binary cache isn't too important.

Allowing end-users to use the binary cache is just be a bonus that comes built into cachix.

Adding it would still reduce your CI usage, which may be nice, but isn't critical or urgent.

For the [...] machine user workflows, I would like to see strong use cases to have them from more Nix users.

The "machine user" stuff mentioned above is not relevant to end-users. It is just a workaround to trigger CI to run against PRs opened by the auto-update workflow.

Without a machine user (or another workaround), you can still trigger CI by checking out the PR branch, doing a no-op --amend commit, and force-pushing.

Or you can manually verify things build ok without running CI, by checking out the PR branch and running nix build.

It's common practice to create a non-bot "machine user" (e.g. @nix-infra-bot) and use their Personal Access Token when pushing up the update PR.

As an aside, is CI being triggered to run against your current dependabot-opened PRs?

TL;DR: PR events by "bot users" do not trigger CI.

Linking relevant upstream issue dependabot/dependabot-core#7340 because it would be nice to have this natively in dependabot core. Besides dependabot, it looks automating dependencies via renovate may be possible.

This issue is also relevant: NixOS/nix#9823

Frankly, I don't think dependabot is likely to git nix flake update support any time soon, as convenient as that would be.

I've not looked at renovate before. I think it's less popular because it's not fully open-source? They link to a couple issues in their documentation (renovatebot/renovate#29721 and renovatebot/renovate#29380), but maybe it'd be fine for your use-case?

Currently most people either use https://github.com/DeterminateSystems/update-flake-lock or implement a custom workflow that runs nix flake update --commit-lock-file and gh pr create (e.g. nixvim's).

IMO this is fine, especially if a machine user is added so that CI gets triggered on the update PRs. But even without that, this is fine as an initial implementation.

@R1kaB3rN
Copy link
Member

Renovate should be doable. And one of the issues you linked should have been closed by renovatebot/renovate#31921 but if we were to pivot I think we'd need to wait for renovatebot/renovate#33991 to be merged at least due to a recent bug when the bot updates the Flake. But again, I think dependabot and the workflows here are sufficient.

@R1kaB3rN R1kaB3rN merged commit dd27f50 into Open-Wine-Components:main Feb 10, 2025
13 checks passed
@MattSturgeon MattSturgeon deleted the ci/nix branch February 10, 2025 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants