Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable support for other ARM GNU Toolchains, add arm-none-linux-gnueabihf toolchain #30

Merged
merged 23 commits into from
Mar 25, 2024

Conversation

AmateurECE
Copy link
Contributor

Hey, there!

Thanks for your great work on this hermetic toolchain. I needed the arm-none-eabi toolchain, as well as the arm-none-linux-gnueabihf toolchain for a project I'm working on, so I decided to extend this repository to make both of them available. This PR also lays the groundwork for easily integrating other ARM GNU toolchains (see 2e0befc for an example of how this might look for additional toolchains).

You may not be interested in integrating this capability into your existing repository. That's totally fine! If that's the case, I'd like to seek your permission to submit my fork to the Bazel Central Registry.

If you are interested in integrating this capability, I've listed some notes that I'd like you to be aware of as you're reviewing this pull request, along with some items for your consideration:

  1. I sought primarily to keep the existing arm-none-eabi API available to both WORKSPACE and bzlmod consumers. Unfortunately, there were a few areas in which I felt this was not possible--the tag classes in the bzlmod extension, as an example.
  2. The new arm-none-linux-gnueabihf API mirrors the existing arm-none-eabi API. Is this desired? If not, are there any API changes you might like to see?
  3. This PR changes the module name to @arm_gnu_toolchain. I am open to ideas if this is not a desirable name, but I do feel that there should be some change from @arm_none_eabi if we are to add support for other toolchains.
  4. There are a few additional things that I would like to do before this PR is merged, if there's an appetite to merge it. For example, I think we should consider replacing the usage of some elements from the @bazel_tools repository with the @rules_cc repository. There is also an issue with the bzlmod extension that does not allow the toolchain version to be overridden in dependent MODULE.bazel files which should be fixed.
  5. I think there's an opportunity for a project like this to add a lot of value to the Bazel community by becoming available in the Bazel Central Registry. Have you considered submitting this repository there? If that's something you're interested in doing, I could potentially provide support in that area. I have no experience doing that, but I do think it would be a worthwhile investment and so I'd be willing to invest my effort.

Thank you very much for your time in reviewing this pull request, and don't hesitate to reach out with any questions you may have!

@hexdae
Copy link
Owner

hexdae commented Mar 11, 2024

Hey, thanks for this contribution, I think it's a good idea to unify some of these toolchains under one roof and submit that to the Bazel registry. Since this repo is getting some traction I was thinking about cleaning it up and uploading.

I think your additions are a good thing to get in before submitting to the BCR. Here's some quick answers to your points:

  1. I think we should focus on Bazel mod support, and only limited WORKSPACE compatibility (if at all). WORKSPACE dependencies were always a bit contrived and anyone on Bazel 6.4.0 and above should be able two use MODULE now
  2. I think we should take a look at projects rules_rust and make sure our APIs are intuitive like theirs. Very open to suggestions here
  3. arm_gnu_toolchain makes sense, we can talk about renaming this repo if we feel it's necessary

4 and 5, we def agree. I also have a set of RISCV GNU toolchains that we could update and push if you care about embedded dev tools

I'll review this PR shortly, but could you take care of fixing the CI issues?

@AmateurECE
Copy link
Contributor Author

CI issues appear to be resolved now! I also resolved the issues outlined in 4 and 5. Unfortunately rules_cc does not yet define ACTION_NAMES.llvm_cov, for some reason, so there is one reference to @bazel_tools remaining.

Very good question about the intuitiveness of the APIs! I think that in general they are, at least there isn't any specific way that I would change them, after reviewing rules_rust. One interesting data point is that while @bazelembedded/bazel-embedded does appear to expose their cc_*_config rules (e.g. clang_toolchain_config) much in the same way as cc_arm_gnu_toolchain_config, they define a number of interesting features that can be enabled, and encourage users to configure the toolchain using those instead of the cc_*_config rules directly. I don't know if I have a preference for one over the other, I think I will need to try them both out more extensively to form an opinion.

Feel free to provide additional comment on my latest changes, but I otherwise feel that this PR is ready to merge!

I think what's next for me might be adding support for the aarch64-none-elf and aarch64-none-linux-gnu toolchains, if you're interested in adding those, and potentially also adding some of the versions that ARM released between 9.2.1 and 13.2.1.

Copy link
Owner

@hexdae hexdae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, this is an excellent PR and it cleans up some of the sharp edges I was worried about.

We just merged a more ergonomic way to handle standard includes by @RyanDraves, if we can rebase on the main branch and include those changes here this should be good to merge (happy to do it given how much work you already added, just let me know if you don't have time)

I think we should add the arm-none-eabi missing versions and then release to Bazel mod before adding support for many other toolchains. I fear scope creep otherwise

"""
_arm_gnu_toolchain(name, toolchain = "arm_none_eabi",
toolchain_prefix = "arm-none-eabi", version = version,
abi_version = "eabi", copts = ["-nostdinc"] + copts,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we just merged a change to optionally disable standard includes (which sometimes are needed for C++ folks), it would be great to use those changes. Happy to patch this otherwise excellent PR if you don't have the time

@AmateurECE
Copy link
Contributor Author

No problem at all! I'll address the C++ standard library includes and the rebase within the next couple of days.

Roger on releasing to BCR before adding support for new toolchains--I'm a frequent perpretrator of the "just one more feature" mindset. 😆

Previously, there was a list of compatible CPUs that were used to form
the target constraints for the toolchain. Now, however, we need to
distinguish toolchains on more than just @platforms//cpu, so manage
these as a collection of constraint values instead.
This commit renames the module to arm_gnu_toolchain from arm_none_eabi.
API-compatibility is _mostly_ retained through the new top-level
repository generation mechanism. Using this module to configure an
arm-none-eabi toolchain results in the generation of an external
repository called @arm_none_eabi. In this way, clients can still rely on
tools and targets that were previously defined by the old module.
Consumers must now import module extensions and repository rules from
the new module, however. See the updated README.md for more information.
Introduce a new custom repository rule to allow us to templatize the
"compiler" BUILD.bazel file (toolchain/compilers{nix,win}.BUILD). This
allows us an easy path to changing out the toolchain prefix when
integrating new toolchain types.
We now encode all toolchain-specific items in the attributes of the
generic "cc_arm_gnu_toolchain_config" rule. This allows us to use this
rule for other ARM GNU toolchains in the future.
All the interesting stuff now lives in a "private" _arm_gnu_toolchain
function that specific toolchains can delegate to. This provide a common
toolchain mechanism that can be used for creating and registering arm
linux toolchains, for example.
Continuing with the trend of "genericization," this commit abstracts
everything specific to the arm-none-eabi platform out of the
arm_none_eabi_deps function into a new function that can be leveraged by
additional toolchains. The arm_none_eabi_deps function is kept to
maintain API stability.
Previously, there was a single tag class for the "arm_gnu" module
extension which would be used by consumers to pull in arm vendored
toolchains. Now, however, the tags map to the particular toolchain
through the toolchain's target name. See the updated README for more
information.
This commit introduces the named toolchain, which functions much in the
same way as the existing arm-none-eabi toolchains. A couple of minor
changes to existing common code are required to get simple
applications to compile.
Previously, the BUILD template for the cross-hosted toolchain external
repository (host-specific) had a defect that caused the paths of tools
to be resolved incorrectly.
Currently, there is an issue where the consuming MODULE.bazel is not
capable of overriding the version of the toolchain specified in our
MODULE.bazel. Create a FIXME comment in extensions.bzl.

Previously, there was also an issue where selecting any toolchain
version besides 9.x.x caused a failure when setting up the external
repository, because the generated BUILD.bazel wasn't provided the
version of the toolchain it was generating.
This commit fixes an issue with the bzlmod extension that prevents
consuming MODULE.bazel files from being able to select the version of
the toolchain. Now, the bzlmod extension selects the minimum version
from the modules that have requested the toolchain and uses that for the
system. This allows consumers to upgrade the version of
arm_gnu_toolchain without needing to ugprade their compiler version, as
well.
This commit combines the two previous BUILD templates for the generated
repos, nix.BUILD and win.BUILD, into one templatized compiler.BUILD
template, and it also moves the other existing templates into the
toolchain/templates folder.
Previously, there was a change that required the version of the
toolchain to be specified in the cc_arm_*_config rules. This isn't
ideal, because it requires consumers to list the version in two places.
So instead, introduce some new filegroup targets in the cross-hosted
toolchain repository that allows these rules to infer the toolchain
version. Also fix some other existing issues with the examples.
Organize the examples by toolchain so that CI workflows can selectively
build examples based on which toolchains are available for the host
platform.
Previously, CI builds were failing for Windows because the arm-linux
toolchain include paths exceeded the Maximum Path Length. Use
--output_user_root in the workflow to avoid this limitation.
This commit gets us almost all the way to using rules_cc instead of
bazel_tools for forward compatibility after the starlarkification of the
C++ rules is complete. The only import that can't currently be changes
is ACTION_NAMES, because ACTION_NAMES.llvm_cov is not defined in
rules_cc as of today.
This affects default arguments and the README, but should not have any
effect on consumers using older toolchains.
@AmateurECE
Copy link
Contributor Author

Alright, and I believe that's it! I set include_std = True for the ARM Linux toolchain--hopefully that's alright. Thanks for your expediency in getting this PR merged, I'm really looking forward to using the recent features in my project!

@hexdae hexdae merged commit 27b0cff into hexdae:master Mar 25, 2024
5 checks passed
@hexdae
Copy link
Owner

hexdae commented Mar 25, 2024

Merged, thanks for the amazing work @AmateurECE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants