[Flang-RT] Make ClangBuilder use LLVM_ENABLE_RUNTIMES=flang-rt

[Meinersbur]

With the option checkout_flang=True, ClangBuilder-based builders also compile Flang. Modify ClangBuilder to use LLVM_ENABLE_RUNTIMES=flang-rt.

Split off from #333

Affected builders:

• clang-aarch64-full-2stage
• clang-aarch64-sve-vla
• clang-aarch64-sve-vla-2stage
• clang-aarch64-sve-vls
• clang-aarch64-sve-vls-2stage
• clang-aarch64-sve2-vla
• clang-aarch64-sve2-vla-2stage
• clang-arm64-windows-msvc
• clang-arm64-windows-msvc-2stage

Affected workers:

• linaro-clang-aarch64-full-2stage
• linaro-g3-01
• linaro-g3-02
• linaro-g3-03
• linaro-g3-04
• linaro-g4-01
• linaro-g4-02
• linaro-armv8-windows-msvc-04
• linaro-armv8-windows-msvc-02

Admins listed for those workers:

• Linaro Toolchain Working Group linaro-toolchain@lists.linaro.org
• Leandro Lupori leandro.lupori@linaro.org
• David Spickett david.spickett@linaro.org

Verified locally on x86_64 (with necessary changes) using the instructions at https://llvm.org/docs/HowToAddABuilder.html#testing-a-builder-config-locally. Once those pass, I will remove the draft status.

[DavidSpickett]

I assume testing this locally is just adding -DLLVM_ENABLE_RUNTIMES=flang-rt? And I assume that currently our bots are not building or using flang-rt in any way.

@ceseo for Linux if you verify this on an SVE2 machine with a 2 stage build, that should cover everything else. Windows will be slower, but Omair might know which machines are the fast ones.

[Meinersbur]

@DavidSpickett That is correct. Without it, FLANG_INCLUDE_RUNTIME default to ON, meaning the old way of building the runtime. With LLVM_ENABLE_RUNTIMES=flang-rt it defaults to OFF getting replaced woth the new way of building the runtime.

[ceseo]

I assume testing this locally is just adding -DLLVM_ENABLE_RUNTIMES=flang-rt? And I assume that currently our bots are not building or using flang-rt in any way.

@ceseo for Linux if you verify this on an SVE2 machine with a 2 stage build, that should cover everything else. Windows will be slower, but Omair might know which machines are the fast ones.

I think @luporl has already done this. I remember him mentioning about this change a few weeks ago.

[Meinersbur]

I do not have aarch64 desktop hardware, so results are on x86_64. It means I can only run those without aarch-specifiic flags.

Could not verify the following for unrelated reasons:

• clang-arm64-windows-msvc (reason: ORC tests failing with error "llvm-jitlink.exe: -orc-runtime requires process symbols", passes "liborc_rt-x86_64.a" to llvm-jitlink.exe which does not exists. Possibly because static library suffix on Windows is .lib)
• clang-arm64-windows-msvc-2stage (reason: [flang] Add cmake error if building with clang-cl and MSVC 17.12 llvm-project#120114)

Successfully verified thiese builders (running on Linux, removing Windows-specifics from builder configuration):

• clang-arm64-windows-msvc
• clang-arm64-windows-msvc-2stage

[luporl]

I assume testing this locally is just adding -DLLVM_ENABLE_RUNTIMES=flang-rt? And I assume that currently our bots are not building or using flang-rt in any way.
@ceseo for Linux if you verify this on an SVE2 machine with a 2 stage build, that should cover everything else. Windows will be slower, but Omair might know which machines are the fast ones.

I think @luporl has already done this. I remember him mentioning about this change a few weeks ago.

@ceseo actually, some weeks ago I just mentioned that we would need to add -DLLVM_ENABLE_RUNTIMES=flang-rt to all of our flang-enabled bots to avoid errors, when the flang-rt changes landed.
Since then @Meinersbur changed the default to build flang-rt when flang is enabled, but it's recommended to use -DLLVM_ENABLE_RUNTIMES=flang-rt.
So it's better to verify this on AArch64, with a 2 stage build, as @DavidSpickett suggested, to confirm everything work as expected.

[Meinersbur]

Since then @Meinersbur changed the default to build flang-rt when flang is enabled, but it's recommended to use -DLLVM_ENABLE_RUNTIMES=flang-rt.

This is not yet the case. If you do not add -DLLVM_ENABLE_RUNTIMES=flang-rt, currenty you build the non-LLVM_ENABLE_RUNTIMES version for the Flang runtime. The change you mentioned will come with #124126.

depends_on_projects=[...,'flang-rt'] is necessary for the buildbots to recognize when to rebuild. Currently a change in the flang-rt/ directory will not trigger a rebuild.

So it's better to verify this on AArch64, with a 2 stage build, as @DavidSpickett suggested, to confirm everything work as expected.

Can you approve the patch then?

[luporl]

This is not yet the case. If you do not add -DLLVM_ENABLE_RUNTIMES=flang-rt, currenty you build the non-LLVM_ENABLE_RUNTIMES version for the Flang runtime. The change you mentioned will come with #124126.

Hmm, interesting, I hadn't noticed we still had the old way of building Flang runtime. So this PR must land before llvm/llvm-project#124126 to avoid breaking the bots right? And we also need to wait until the buildmaster picks it.

depends_on_projects=[...,'flang-rt'] is necessary for the buildbots to recognize when to rebuild. Currently a change in the flang-rt/ directory will not trigger a rebuild.

Just to confirm, with this PR, that adds flang-rt to depends_on_projects, -DLLVM_ENABLE_RUNTIMES=flang-rt will be added to the cmake command for all builders that have checkout_flang=True and nothing else will need to be done on the buildbot side, right?

So it's better to verify this on AArch64, with a 2 stage build, as @DavidSpickett suggested, to confirm everything work as expected.

Can you approve the patch then?

As this affects many buildbots, and IIRC the buildmaster is updated only once a week, we really want to check that it works as expected, at least on AArch64/Linux, in addition to x86_64 that you have already tested, to avoid having many bots broken for a week.

[Meinersbur]

Hmm, interesting, I hadn't noticed we still had the old way of building Flang runtime. So this PR must land before llvm/llvm-project#124126 to avoid breaking the bots right? And we also need to wait until the buildmaster picks it.

This was the intention.

1. Commit [Flang] LLVM_ENABLE_RUNTIMES=flang-rt llvm-project#110217 which is adding a new build option to build the flang-rt (think of introducing LLVM_ENABLE_RUNTIMES=libcxx in addition to the already existing LLVM_ENABLE_PROJECTS=libcxx)
2. Update all the buildbots to use the new configuration
3. Commit [Flang] Remove FLANG_INCLUDE_RUNTIME llvm-project#124126 to remove the old build instructions (think of removing LLVM_ENABLE_PROJECTS=libcxx, as happened in llvm/llvm-project@258477e)

With the planned addition to llvm/llvm-project#124126 of implicitly adding LLVM_ENABLE_RUNTIMES=flang-rt when LLVM_ENABLE_PROJECTS=flang is present, most builders should still work. But I'd rather make that change explicitly for the buildbot to identify problems beforehand. Builders that would still break are the out-of-tree builders (e.g. flang-aarch64-out-of-tree), since without LLVM_ENABLE_PROJECTS=flang present, of course LLVM_ENABLE_RUNTIMES=flang-rt will also not be present (nor would it work in abscence of a Fortran compiler). Patch for out-of-tree builders in preparation.

Just to confirm, with this PR, that adds flang-rt to depends_on_projects, -DLLVM_ENABLE_RUNTIMES=flang-rt will be added to the cmake command for all builders that have checkout_flang=True and nothing else will need to be done on the buildbot side, right?

Correct. Also see my response here: #383 (comment)

As this affects many buildbots, and IIRC the buildmaster is updated only once a week, we really want to check that it works as expected, at least on AArch64/Linux, in addition to x86_64 that you have already tested, to avoid having many bots broken for a week.

I understand, its not urgent so take your time.