This change modifies the LLVM build process to disable ccache by default
and only enable it when the --use-ccache flag is explicitly passed. This
reduces CI storage consumption while still allowing developers to opt-in
to ccache for faster incremental builds.
Changes:
- Modified build_llvm() to accept use_ccache parameter (defaults to False)
- Updated ccache logic to only enable when use_ccache=True and platform is not Windows
- Pass options.use_ccache from main() to build_llvm() call
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add a new --use-ccache flag that will allow developers to opt-in to using
ccache for LLVM builds. This is part of a larger effort to disable ccache
by default to reduce CI storage consumption while still allowing local
developers to benefit from faster incremental builds.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
compilation on macos / build_samples_wasm_c_api (${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, $AOT_BUILD_OPTIONS, macos-15) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, $FAST_INTERP_BUILD_OPTIONS, macos-15) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, $AOT_BUILD_OPTIONS, macos-15-intel) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, $FAST_INTERP_BUILD_OPTIONS, macos-15-intel) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_MULTI_MODULE=0, ${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, $LLVM_EAGER_JIT_BUILD_OPTIONS, macos-15) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_MULTI_MODULE=0, ${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, $LLVM_LAZY_JIT_BUILD_OPTIONS, macos-15) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_MULTI_MODULE=0, ${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, $LLVM_EAGER_JIT_BUILD_OPTIONS, macos-15-intel) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_MULTI_MODULE=0, ${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, $LLVM_LAZY_JIT_BUILD_OPTIONS, macos-15-intel) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_SIMD=0, ${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, $CLASSIC_INTERP_BUILD_OPTIONS, macos-15) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api (-DWAMR_BUILD_SIMD=0, ${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, $CLASSIC_INTERP_BUILD_OPTIONS, macos-15-intel) (push) Has been cancelled
compilation on macos / build_samples_others (${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, macos-15-intel) (push) Has been cancelled
compilation on SGX / spec_test_default (${{ needs.build_llvm_libraries.outputs.cache_key }}, ubuntu-22.04, aot, $DEFAULT_TEST_OPTIONS) (push) Has been cancelled
compilation on SGX / spec_test_default (${{ needs.build_llvm_libraries.outputs.cache_key }}, ubuntu-22.04, aot, $SIMD_TEST_OPTIONS) (push) Has been cancelled
compilation on SGX / spec_test_default (${{ needs.build_llvm_libraries.outputs.cache_key }}, ubuntu-22.04, aot, $XIP_TEST_OPTIONS) (push) Has been cancelled
compilation on SGX / spec_test_default (${{ needs.build_llvm_libraries.outputs.cache_key }}, ubuntu-22.04, classic-interp, $DEFAULT_TEST_OPTIONS) (push) Has been cancelled
compilation on SGX / spec_test_default (${{ needs.build_llvm_libraries.outputs.cache_key }}, ubuntu-22.04, fast-jit, $DEFAULT_TEST_OPTIONS) (push) Has been cancelled
* fix: disable unsigned integer overflow sanitization in build configurations
FYI: from https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
`-fsanitize=unsigned-integer-overflow`: Unsigned integer overflow, where the result of an unsigned integer computation cannot be represented in its type. Unlike signed integer overflow, this is not undefined behavior, but it is often unintentional. This sanitizer does not check for lossy implicit conversions performed before such a computation.
It brings a more common question: which is better, pre-additional-check or post-additional-check to fix a potential unsigned integer overflow? A pre-additional-check involves using a check to prevent integer overflow from the very beginning. A post-additional-check involves using a check after addition to see if there is an overflow.
In this project, post-additional-checking is widely used. let's follow the routine.
for performance sensitive logic, use __builtin_add_overflow etc. provide something like 9a5622791e/lib/platform.h (L176-L191) and encourage the use of them.
ref. https://github.com/bytecodealliance/wasm-micro-runtime/pull/4549#issuecomment-3218687294
* fix: ensure proper definition checks for build options in CMakeLists of wasm-mutator
* optimize how to involve sanitizer flags
* fix: update LLVM branch and refine sanitizer flags in CMake configurations
* fix: add requests package to development requirements
compilation on macos / build_samples_wasm_c_api ($CLASSIC_INTERP_BUILD_OPTIONS, macos-13, https://github.com/WebAssembly/wabt/releases/download/1.0.31/wabt-1.0.31-macos-12.tar.gz, https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-macos.tar.gz) (push) Has been cancelled
compilation on macos / build_samples_wasm_c_api ($FAST_INTERP_BUILD_OPTIONS, macos-13, https://github.com/WebAssembly/wabt/releases/download/1.0.31/wabt-1.0.31-macos-12.tar.gz, https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-macos.tar.gz) (push) Has been cancelled
compilation on macos / build_samples_others (${{ needs.build_llvm_libraries_on_arm_macos.outputs.cache_key }}, macos-14, https://github.com/WebAssembly/wabt/releases/download/1.0.31/wabt-1.0.31-macos-12.tar.gz, https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-s… (push) Has been cancelled
compilation on macos / build_samples_others (${{ needs.build_llvm_libraries_on_intel_macos.outputs.cache_key }}, macos-13, https://github.com/WebAssembly/wabt/releases/download/1.0.31/wabt-1.0.31-macos-12.tar.gz, https://github.com/WebAssembly/wasi-sdk/releases/download/wasi… (push) Has been cancelled
- For Windows, llvm libs need to cache more directories, so use a multi-line
environment variable for paths
- Remove conditionally build directories `win32build`, just use `build` for all platform
- Add Windows wamrc and iwasm(disable lib pthread semaphore and fast jit for now)
build in release CI
LLVM PGO (Profile-Guided Optimization) allows the compiler to better optimize code
for how it actually runs. This PR implements the AOT static PGO, and is tested on
Linux x86-64 and x86-32. The basic steps are:
1. Use `wamrc --enable-llvm-pgo -o <aot_file_of_pgo> <wasm_file>`
to generate an instrumented aot file.
2. Compile iwasm with `cmake -DWAMR_BUILD_STATIC_PGO=1` and run
`iwasm --gen-prof-file=<raw_profile_file> <aot_file_of_pgo>`
to generate the raw profile file.
3. Run `llvm-profdata merge -output=<profile_file> <raw_profile_file>`
to merge the raw profile file into the profile file.
4. Run `wamrc --use-prof-file=<profile_file> -o <aot_file> <wasm_file>`
to generate the optimized aot file.
5. Run the optimized aot_file: `iwasm <aot_file>`.
The test scripts are also added for each benchmark, run `test_pgo.sh` under
each benchmark's folder to test the AOT static pgo.
Upgrade the version of related toolkits:
- upgrade llvm to 15.0
- upgrade wasi-sdk to 19.0
- upgrade emsdk to 3.1.28
- upgrade wabt to 1.0.31
- upgrade binaryen to 111
And upgrade the CI scripts, sample workload build scripts, Dockerfiles, and documents.
Enhance workflows by caching llvm directories directly instead of
caching the llvm-xxx.tar.gz packages, so as to reduce the time
consumption of unpacking the cached LLVM packages.
Implement the latest SIMD opcodes and update LLVM 13.0,
update the llvm build scripts, update the sample workloads‘ build scripts,
and build customized wasi-sdk to build some workloads.
Also refine the CI rules.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
