Use sccache to accelerate android build #9587

kirklandsign · 2025-03-25T17:28:17Z

Android didn't use sccache because it has its own NDK clang. Now use env var to trigger clang from sccache.

pytorch-bot · 2025-03-25T17:28:20Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9587

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 3 Unrelated Failures

As of commit 0ce7cc2 with merge base 7248b19 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Command docker exec -t 69e35649119e2387c1f3f8c66584ebdc550954fc56de6e41b36a24b95896ba62 /exec failed with exit code 2
pull / test-llama-runner-linux-android / linux-job (gh)
RuntimeError: Command docker exec -t 5d82678f077e41a832c663cecde0f43231bc1e95fdcc536c532fffdea6422b1f /exec failed with exit code 2

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-llava-runner-linux / linux-job (gh) (trunk failure)
test_generated_output
pull / unittest / macos / macos-job (gh) (trunk failure)
backends/xnnpack/test/passes/test_convert_to_linear.py::TestConvertToLinear::test_fp32_convert_to_linear
pull / unittest-editable / macos / macos-job (gh) (trunk failure)
backends/xnnpack/test/ops/test_check_quant_params.py::TestCheckQuantParams::test_inject_invalid_zp

This comment was automatically generated by Dr. CI and updates every 15 minutes.

jathu · 2025-03-26T18:38:11Z

.github/workflows/_android.yml

+        # Use sccache for NDK compiler as well
+        export CMAKE_CXX_COMPILER_LAUNCHER=sccache
+        export CMAKE_C_COMPILER_LAUNCHER=sccache


What are your thoughts on exporting this in .ci/scripts/setup-linux.sh so we get it for all CI builds?

I would rely on https://github.com/pytorch/executorch/blob/main/.ci/docker/common/install_cache.sh#L47 to trigger sccache for general linux.

Seems that if I put these in https://github.com/pytorch/executorch/blob/main/.ci/scripts/setup-linux.sh, it will be used not only on android, but generic, and I would avoid doing that to mess up with the set up above.

Generally, I'm hoping we can find one place to change and export these variables instead of multiple places. If someone adds another android job, they do not get caching if they forget these lines

IMO, you could try the simpler fix to install NDK before sccache in https://github.com/pytorch/executorch/blob/main/.ci/docker/ubuntu/Dockerfile#L57-L83. We install sccache as part of building Docker image and /opt/cache/bin is added to PATH in https://github.com/pytorch/executorch/blob/main/.ci/docker/common/install_cache.sh#L43. Atm, installing NDK happens after installing sccache, so I guess NDK path takes the priority?

Sorry not sure if that's the situation here. Currently for android build, clang path is /opt/ndk/toolchains/llvm/prebuilt/...bin and we should not use /opt/cache/bin/clang (stub which points to the system clang)?

Basically for android, we need /opt/ndk/.../clang to point to a stub as well?

Using a stub works transparently as part of the Docker image. Setting CMAKE_CXX_COMPILER_LAUNCHER needs to be done later, i.e. people need to remember to set that explicitly.

Honestly I feel like it's hard to do stub... In NDK we have so many variants of clang

armv7a-linux-androideabi21-clang i686-linux-android29-clang++ x86_64-linux-android21-clang armv7a-linux-androideabi21-clang++ i686-linux-android30-clang x86_64-linux-android21-clang++ armv7a-linux-androideabi22-clang i686-linux-android30-clang++ x86_64-linux-android22-clang armv7a-linux-androideabi22-clang++ i686-linux-android31-clang x86_64-linux-android22-clang++ armv7a-linux-androideabi23-clang i686-linux-android31-clang++ x86_64-linux-android23-clang armv7a-linux-androideabi23-clang++ i686-linux-android32-clang x86_64-linux-android23-clang++

@kirklandsign I think you can use $ANDROID_NDK/ndk-which clang to get the specific location

@jathu I can have a try - just use clang instead of these variants (which are basically for clang --target=x86_64-linux-android21 stuff)

I was just worried if somehow android NDK triggered clang via x86_64-linux-android21-clang not clang

@huydhn @jathu I feel like for android it's still hard to get the stub working. See error like https://github.com/pytorch/executorch/actions/runs/14115338695/job/39544105035?pr=9587 for my version f2e12cb#diff-0a862d15a4e6dff44196719a594b983aece596c7dc38eba416d83e12a95206e6R92.

kirklandsign · 2025-03-27T17:32:47Z

.ci/docker/common/install_cache.sh

@@ -89,10 +89,26 @@ init_sccache() {
  as_ci_user sccache --zero-stats || true
 }

+function write_android_sccache_stub() {
+  BINARY=$1
+  mv "/opt/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/$1" "/opt/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/.$1"


/opt/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang
/opt/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++

[TEST] Try to see if sccache is in use

dc12aaf

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 25, 2025

kirklandsign and others added 4 commits March 25, 2025 14:01

Merge branch 'main' into test-sccache

96d9c96

Update _android.yml

b15ddf1

Update pull.yml

70e5736

Trigger sccache

abb5bc9

kirklandsign added the topic: not user facing label Mar 26, 2025

Update

c3465a3

kirklandsign marked this pull request as ready for review March 26, 2025 18:29

kirklandsign changed the title ~~[TEST] Try to see if sccache is in use~~ Use sccache to accelerate android build Mar 26, 2025

kirklandsign requested review from jathu and larryliu0820 March 26, 2025 18:31

jathu reviewed Mar 26, 2025

View reviewed changes

kirklandsign temporarily deployed to upload-benchmark-results March 26, 2025 19:21 — with GitHub Actions Inactive

Use stub in docker

f2e12cb

kirklandsign commented Mar 27, 2025

View reviewed changes

kirklandsign added 2 commits March 27, 2025 11:03

Update install_cache.sh

bb2ad5b

Update install_cache.sh

0ce7cc2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use sccache to accelerate android build #9587

Use sccache to accelerate android build #9587

kirklandsign commented Mar 25, 2025 •

edited

Loading

pytorch-bot bot commented Mar 25, 2025 •

edited

Loading

jathu Mar 26, 2025

kirklandsign Mar 26, 2025

jathu Mar 26, 2025

huydhn Mar 26, 2025

kirklandsign Mar 26, 2025

huydhn Mar 26, 2025

kirklandsign Mar 26, 2025

jathu Mar 26, 2025

kirklandsign Mar 27, 2025

kirklandsign Mar 28, 2025

kirklandsign Mar 27, 2025

Use sccache to accelerate android build #9587

Are you sure you want to change the base?

Use sccache to accelerate android build #9587

Conversation

kirklandsign commented Mar 25, 2025 • edited Loading

pytorch-bot bot commented Mar 25, 2025 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9587

❌ 2 New Failures, 3 Unrelated Failures

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kirklandsign commented Mar 25, 2025 •

edited

Loading

pytorch-bot bot commented Mar 25, 2025 •

edited

Loading