Adding a new cmake flag (cache variable) `WAMR_BUILD_MEMORY64` to enable
the memory64 feature, it can only be enabled on the 64-bit platform/target and
can only use software boundary check. And when it is enabled, it can support both
i32 and i64 linear memory types. The main modifications are:
- wasm loader & mini-loader: loading and bytecode validating process
- wasm runtime: memory instantiating process
- classic-interpreter: wasm code executing process
- Support memory64 memory in related runtime APIs
- Modify main function type check when it's memory64 wasm file
- Modify `wasm_runtime_invoke_native` and `wasm_runtime_invoke_native_raw` to
handle registered native function pointer argument when memory64 is enabled
- memory64 classic-interpreter spec test in `test_wamr.sh` and in CI
Currently, it supports memory64 memory wasm file that uses core spec
(including bulk memory proposal) opcodes and threads opcodes.
ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091https://github.com/bytecodealliance/wasm-micro-runtime/pull/3240https://github.com/bytecodealliance/wasm-micro-runtime/pull/3260
The PR #3259 reverted PR #3192, it fixes#3210 but makes #3170 failed again.
The workaround is that we should update `ctx->dynamic_offset` only for opcode br
and should not update it for opcode br_if. This PR fixes both issue #3170 and #3210.
Some environment may call wasm_runtime_full_init/wasm_runtime_init multiple
times without knowing that runtime is initialized or not, it is better to add lock
and increase reference count during initialization.
ps. https://github.com/bytecodealliance/wasm-micro-runtime/discussions/3253.
Add cmake variable `-DWAMR_BUILD_AOT_INTRINSICS=1/0` to enable/disable
the aot intrinsic functions, which are normally used by AOT XIP feature, and
can be disabled to reduce the aot runtime binary size.
And refactor the code in aot_intrinsics.h/.c.
Should not update `ctx->dynamic_offset` in emit_br_info, since the `Part e` only
sets the dst offsets, the operand stack should not be changed, e.g., the stack
operands are to be used by the opcodes followed by `br_if` opcode.
Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3210.
- Fix nightly run CI failure which was introduced by PR #3239 and now it must set
WAMR_BUILD_TARGET when building iwasm for Android platform
- Remove building llvm, wamrc and jit in CodeQL CI, since it will
do static code analyzing for llvm project and cause CodeQL run
failed: `Oops! A fatal internal error occurred.
This particular kind of error most often happens as a side effect of running out of
disk space.`
This PR fixes the random failing test case `nofollow_errors` mentioned in
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3222
```C
// dirfd: This is the file descriptor of the directory relative to which the pathname is interpreted.
int openat(int dirfd, const char *pathname, int flags, ...);
```
The value should be a directory handle instead of a file handle (which is always -1 in this context)
returned from `openat`.
- Change `C.long` to `C.int64_t` due to error:
```sh
./module.go:119:64: cannot use _Ctype_long(stdinfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
./module.go:120:43: cannot use _Ctype_long(stdoutfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
./module.go:120:60: cannot use _Ctype_long(stderrfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
```
- Change offset from `uint32` to `uint64` due to casting error
ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3220https://stackoverflow.com/questions/70243683/how-to-convert-c-uint64-t-to-cgo-consistently-across-os
Enhance CodeQL Code Security Analysis:
- Add more compilation combinations to build iwasm with different kinds of features
- Disable run on PR created and keep nightly run, since the whole time is very long,
and will check how to restore run on PR created in the future
Add CodeQL Workflow for Code Security Analysis
This pull request introduces a CodeQL workflow to enhance the security analysis of our repository.
CodeQL is a powerful static analysis tool that helps identify and mitigate security vulnerabilities in
our codebase. By integrating this workflow into our GitHub Actions, we can proactively identify
and address potential issues before they become security threats.
We added a new CodeQL workflow file (.github/workflows/codeql.yml) that
- Runs on nightly-run, and consider runs on every pull request to the main branch in the future.
- Excludes queries with a high false positive rate or low-severity findings.
- Does not display results for third-party code, focusing only on our own codebase.
Testing:
To validate the functionality of this workflow, we have run several test scans on the codebase and
reviewed the results. The workflow successfully compiles the project, identifies issues, and provides
actionable insights while reducing noise by excluding certain queries and third-party code.
Deployment:
Once this pull request is merged, the CodeQL workflow will be active and automatically run on
every push and pull request to the main branch. To view the results of these code scans, please
follow these steps:
1. Under the repository name, click on the Security tab.
2. In the left sidebar, click Code scanning alerts.
Additional Information:
- You can further customize the workflow to adapt to your specific needs by modifying the workflow file.
- For more information on CodeQL and how to interpret its results, refer to the GitHub documentation
and the CodeQL documentation.
Signed-off-by: Brian <bayuan@purdue.edu>
The symbols in windows 32-bit may start with '_' and can not be found
when resolving the relocations to them. This PR ignores the underscore
when handling the relocation name of AOT_FUNC_INTERNAL_PREFIX, and
redirect the relocation with name "_aot_stack_sizes" to the relocation with
name ".aot_stack_sizes" (the name of the data section created).
ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3216
- Merge unused field `used_to_be_wasi_ctx` in `AOTModuleInstance` into `reserved` area
- Add field `memory_lock` in `WASMMemoryInstance` for future refactor
- Go binding: fix type error
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3220
- Python binding:
type annotation uses the union operator "|", which requires Python version >=3.10
This allows to know the beginning of the wasm address space. At the moment
to achieve that, we need to apply a `hack wasm_runtime_addr_app_to_native(X)-X`
to get the beginning of WASM memory in the nativ code, but I don't see a good
reason why not to allow zero address as a parameter value for this function.
Before PR compilation failed because of two errors:
1 - Usage of CMake target_link_libraries()
Make Error at CMakeLists.txt:154 (target_link_libraries):
The keyword signature for target_link_libraries has already been used with
the target "iwasm_shared". All uses of target_link_libraries with a target
must be either all-keyword or all-plain.
The uses of the keyword signature are here:
* CMakeLists.txt:148 (target_link_libraries)
See https://stackoverflow.com/questions/47737558/uses-of-target-link-libraries-must-be-either-all-keyword-or-all-plain
Fixed by adding keyword INTERFACE
2 - Undefined symbols during linkage, fixed by adding -lwsock32.
Last bit missing from #3206: demangling of function names (useful for wasm
generated from Rust for instance) using `llvm-cxxfilt`.
Before this PR:
```text
0: abort
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/panic_abort/src/lib.rs:85
1: _ZN3std3sys4wasi14abort_internal17h50698daab05bf73bE
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/wasi/mod.rs:181
2: _ZN3std7process5abort17h6bc522b6749f17cfE
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/process.rs:2278
3: _ZN3std5alloc8rust_oom17h452ad5ba6cebff96E
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:364
```
After:
```text
0: abort
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/panic_abort/src/lib.rs:85
1: std::sys::wasi::abort_internal::h50698daab05bf73b
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/wasi/mod.rs:181
2: std::process::abort::h6bc522b6749f17cf
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/process.rs:2278
3: std::alloc::rust_oom::h452ad5ba6cebff96
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:364
```
Update the `addr2line` script so that:
- line info is printed in a more convenient format, e.g.
```
0: c
at wasm-micro-runtime/test-tools/addr2line/trap.c:5:1
1: b
at wasm-micro-runtime/test-tools/addr2line/trap.c:11:12
2: a
at wasm-micro-runtime/test-tools/addr2line/trap.c:17:12
```
similar to how Rust prints stack traces when there's a panic. In an IDE, the user
can conveniently click on the printed path and be redirected to the file line.
- a new `--no-addr` argument can be provided to the script
It can be used in fast interpreter mode (that is not supported by the script otherwise)
or with older wamr versions (where the stack trace only had the function index info
and not the function address). In that case, `wasm-objdump` is used to get the function
name from the index and `llvm-dwarfdump` to obtain the location info (where the line
refers to the start of the function).
Merge branch dev/rust_sdk to main.
The wamr-rust-sdk has been migrated to the standalone repo:
https://github.com/bytecodealliance/wamr-rust-sdk
So here we just update the document to add the introduction.
- Address values in call stack dump are relative to file beginning
- If running under fast-interp mode, address values are relative to
every pre-compiled function beginning, which is not compatible
with addr2line
This PR adds a max_memory_pages parameter to module instantiation APIs,
to allow overriding the max memory defined in the WASM module.
Sticking to the max memory defined in the module is quite limiting when
using shared memory in production. If targeted devices have different
memory constraints, many wasm files have to be generated with different
max memory values. And device constraints may not be known in advance.
Being able to set the max memory value during module instantiation allows
to reuse the same wasm module, e.g. by retrying instantiation with different
max memory value.
If the language is not specified, CMake will try to find C++ compiler, even
though it is not really needed in that case (as the project is only written in C).
Now that the filesystem implementation is now complete, the previous
test filters on Windows can be removed. Some of the tests only pass when
certain environment variables have been set on Windows so an extra step
has been added in the wasi test runner script to modify the test config
files before the tests begin.
The stack profiler `aot_func#xxx` calls the wrapped function of `aot_func_internal#xxx`
by using symbol reference, but in some platform like xtensa, it’s translated into a native
long call, which needs to resolve the indirect address by relocation and breaks the XIP
feature which requires the eliminating of relocation.
The solution is to change the symbol reference into an indirect call through the lookup
table, the code will be like this:
```llvm
call_wrapped_func: ; preds = %stack_bound_check_block
%func_addr1 = getelementptr inbounds ptr, ptr %func_ptrs_ptr, i32 75
%func_tmp2 = load ptr, ptr %func_addr1, align 4
tail call void %func_tmp2(ptr %exec_env)
ret void
```
