In some scenarios there may be lots of callings to AOT/JIT functions from the
host embedder, which expects good performance for the calling process, while
in the current implementation, runtime calls the wasm_runtime_invoke_native
to prepare the array of registers and stacks for the invokeNative assemble code,
and the latter then puts the elements in the array to physical registers and
native stacks and calls the AOT/JIT function, there may be many data copying
and handlings which impact the performance.
This PR registers some quick AOT/JIT entries for some simple wasm signatures,
and let runtime call the entry to directly invoke the AOT/JIT function instead of
calling wasm_runtime_invoke_native, which speedups the calling process.
We may extend the mechanism next to allow the developer to register his quick
AOT/JIT entries to speedup the calling process of invoking the AOT/JIT functions
for some specific signatures.
Add an API to set segue flags for wasm-c-api LLVM JIT mode:
```C
wasm_config_t *
wasm_config_set_segue_flags(wasm_config_t *config, uint32 segue_flags);
```
And refactor the original perf support
- use WAMR_BUILD_LINUX_PERF as the cmake compilation control
- use WASM_ENABLE_LINUX_PERF as the compiler macro
- use `wamrc --enable-linux-perf` to generate aot file which contains fp operations
- use `iwasm --enable-linux-perf` to create perf map for `perf record`
The host embedder may also want to terminate the wasm instance
for single-threading mode, and it should work by setting exception
to the wasm instance.
Support new a wasm_config_t, set allocation and linux_perf_support
options to it, and then pass it to wasm_engine_new_with_config to
new an engine with private configuration.
Add an extra argument `os_file_handle file` for `os_mmap` to support
mapping file from a file fd, and remove `os_get_invalid_handle` from
`posix_file.c` and `win_file.c`, instead, add it in the `platform_internal.h`
files to remove the dependency on libc-wasi.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
To run it locally:
```bash
export TSAN_OPTIONS=suppressions=<path_to_tsan_suppressions.txt>
./test_wamr.sh <your flags> -T tsan
```
An example for wasi-threads would look like:
```bash
export TSAN_OPTIONS=suppressions=<path_to_tsan_suppressions.txt>
./test_wamr.sh -w -s wasi_certification -t fast-interp -T tsan
```
Split memory instance's field `uint32 ref_count` into `bool is_shared_memory`
and `uint16 ref_count`, and lock the memory only when `is_shared_memory`
flag is true, no need to acquire a lock for non-shared memory when shared
memory feature is enabled.
Avoid repeatedly initializing the shared memory data when creating the child
thread in lib-pthread or lib-wasi-threads.
Add shared memory lock when accessing some fields of the memory instance
if the memory instance is shared.
Init shared memory's memory_data_size/memory_data_end fields according to
the current page count but not max page count.
Add wasm_runtime_set_mem_bound_check_bytes, and refine the error message
when shared memory flag is found but the feature isn't enabled.
This patch enables mapping host directories to guest directories by parsing
the `map_dir_list` argument in API `wasm_runtime_init_wasi` for libc-wasi. It
follows the format `<guest-path>::<host-path>`.
It also adds argument `--map-dir=<guest::host>` argument for `iwasm`
common line tool, and allows to add multiple mappings:
```bash
iwasm --map-dir=<guest-path1::host-path1> --map-dir=<guest-path2::host-path2> ...
```
To allow non-POSIX platforms such as Windows to support WASI libc
filesystem functionality, create a set of wrapper functions which provide a
platform-agnostic interface to interact with the host filesystem. For now,
the Windows implementation is stubbed but this will be implemented
properly in a future PR. There are no functional changes in this change,
just a reorganization of code to move any direct POSIX references out of
posix.c in the libc implementation into posix_file.c under the shared
POSIX sources.
See https://github.com/bytecodealliance/wasm-micro-runtime/issues/2495 for a
more detailed overview of the plan to port the WASI libc filesystem to Windows.
Avoid the stack traces getting mixed up together when multi-threading is enabled
by using exception_lock/unlock in dumping the call stacks.
And remove duplicated call stack dump in wasm_application.c.
Also update coding guideline CI to fix the clang-format-12 not found issue.
Support muti-module for AOT mode, currently only implement the
multi-module's function import feature for AOT, the memory/table/
global import are not implemented yet.
And update wamr-test-suites scripts, multi-module sample and some
CIs accordingly.
Send a signal whose handler is no-op to a blocking thread to wake up
the blocking syscall with either EINTR equivalent or partial success.
Unlike the approach taken in the `dev/interrupt_block_insn` branch (that is,
signal + longjmp similarly to `OS_ENABLE_HW_BOUND_CHECK`), this PR
does not use longjmp because:
* longjmp from signal handler doesn't work on nuttx
refer to https://github.com/apache/nuttx/issues/10326
* the singal+longjmp approach may be too difficult for average programmers
who might implement host functions to deal with
See also https://github.com/bytecodealliance/wasm-micro-runtime/issues/1910
Add API wasm_runtime_terminate to terminate a module instance
by setting "terminated by user" exception to the module instance.
And update the product-mini of posix platforms.
Note: this doesn't work for some situations like blocking system calls.
This fixes a few test cases in wasi-threads testsuite like wasi_threads_return_main_block.
And also move the special handling for "wasi proc exit" to a more appropriate place.
Introduce module instance context APIs which can set one or more contexts created
by the embedder for a wasm module instance:
```C
wasm_runtime_create_context_key
wasm_runtime_destroy_context_key
wasm_runtime_set_context
wasm_runtime_set_context_spread
wasm_runtime_get_context
```
And make libc-wasi use it and set wasi context as the first context bound to the wasm
module instance.
Also add samples.
Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/2460.
When embedding WAMR, this PR allows to register a callback that is
invoked when memory.grow fails.
In case of memory allocation failures, some languages allow to handle
the error (e.g. by checking the return code of malloc/calloc in C), some
others (e.g. Rust) just panic.
When AOT out of bound linear memory access or stack overflow occurs, the call stack of
AOT functions cannot be unwound currently, so from the exception handler, runtime
cannot jump back into the place that calls the AOT function.
We temporarily skip the current instruction and let AOT code continue to run and return
to caller as soon as possible. And use the zydis library the decode the current instruction
to get its size.
And remove using RtlAddFunctionTable to register the AOT functions since it doesn't work
currently.
## Context
Some native libraries may want to explicitly delete an externref object without
waiting for the module instance to be deleted.
In addition, it may want to add a cleanup function.
## Proposed Changes
Implement:
* `wasm_externref_objdel` to explicitly delete an externeref'd object.
* `wasm_externref_set_cleanup` to set a cleanup function that is called when
the externref'd object is deleted.
- Inherit shared memory from the parent instance, instead of
trying to look it up by the underlying module. The old method
works correctly only when every cluster uses different module.
- Use reference count in WASMMemoryInstance/AOTMemoryInstance
to mark whether the memory is shared or not
- Retire WASMSharedMemNode
- For atomic opcode implementations in the interpreters, use
a global lock for now
- Update the internal API users
(wasi-threads, lib-pthread, wasm_runtime_spawn_thread)
Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/1962
And return ENOSYS. We do that so we can at least compile the code on CI.
We'll be gradually enabling more and more functions.
Also, enabled `proc_raise()` for windows.
There is no standard `realpath` function in the C/C++ standard libraries for Windows,
use `_fullpath` function instead to get absolute path of a directory.
We have observed a significant performance degradation after merging
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1991
Instead of protecting suspend flags with a mutex, we implement the flags
as atomic variable and only use mutex when atomics are not available
on a given platform.
Allow to use `cmake -DWAMR_CONFIGURABLE_BOUNDS_CHECKS=1` to
build iwasm, and then run `iwasm --disable-bounds-checks` to disable the
memory access boundary checks.
And add two APIs:
`wasm_runtime_set_bounds_checks` and `wasm_runtime_is_bounds_checks_enabled`
LLVM PGO (Profile-Guided Optimization) allows the compiler to better optimize code
for how it actually runs. This PR implements the AOT static PGO, and is tested on
Linux x86-64 and x86-32. The basic steps are:
1. Use `wamrc --enable-llvm-pgo -o <aot_file_of_pgo> <wasm_file>`
to generate an instrumented aot file.
2. Compile iwasm with `cmake -DWAMR_BUILD_STATIC_PGO=1` and run
`iwasm --gen-prof-file=<raw_profile_file> <aot_file_of_pgo>`
to generate the raw profile file.
3. Run `llvm-profdata merge -output=<profile_file> <raw_profile_file>`
to merge the raw profile file into the profile file.
4. Run `wamrc --use-prof-file=<profile_file> -o <aot_file> <wasm_file>`
to generate the optimized aot file.
5. Run the optimized aot_file: `iwasm <aot_file>`.
The test scripts are also added for each benchmark, run `test_pgo.sh` under
each benchmark's folder to test the AOT static pgo.
Segue is an optimization technology which uses x86 segment register to store
the WebAssembly linear memory base address, so as to remove most of the cost
of SFI (Software-based Fault Isolation) base addition and free up a general
purpose register, by this way it may:
- Improve the performance of JIT/AOT
- Reduce the footprint of JIT/AOT, the JIT/AOT code generated is smaller
- Reduce the compilation time of JIT/AOT
This PR uses the x86-64 GS segment register to apply the optimization, currently
it supports linux and linux-sgx platforms on x86-64 target. By default it is disabled,
developer can use the option below to enable it for wamrc and iwasm(with LLVM
JIT enabled):
```bash
wamrc --enable-segue=[<flags>] -o output_file wasm_file
iwasm --enable-segue=[<flags>] wasm_file [args...]
```
`flags` can be:
i32.load, i64.load, f32.load, f64.load, v128.load,
i32.store, i64.store, f32.store, f64.store, v128.store
Use comma to separate them, e.g. `--enable-segue=i32.load,i64.store`,
and `--enable-segue` means all flags are added.
Acknowledgement:
Many thanks to Intel Labs, UC San Diego and UT Austin teams for introducing this
technology and the great support and guidance!
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Co-authored-by: Vahldiek-oberwagner, Anjo Lucas <anjo.lucas.vahldiek-oberwagner@intel.com>
Add nightly (UTC time) checks with asan and ubsan, and also put gcc-4.8 build
to nightly run since we don't need to run it with every PR.
Co-authored-by: Maksim Litskevich <makslit@amazon.co.uk>
Fix issue reported in #2172: wasm-c-api `wasm_func_call` may use a wrong exec_env
when multi-threading is enabled, with error "invalid exec env" reported
Fix issue reported in #2149: main instance's `c_api_func_imports` are not passed to
the counterpart of new thread's instance in wasi-threads mode
Fix issue of invalid size calculated to copy `c_api_func_imports` in pthread mode
And refactor the code to use `wasm_cluster_dup_c_api_imports` to copy the
`c_api_func_imports` to new thread for wasi-threads mode and pthread mode.
In multi-threading, this line will eventually call `wasm_cluster_wait_for_all_except_self`:
`DEINIT_VEC(store->instances, wasm_instance_vec_delete)`
As the threads are joining they can call `wasm_interp_dump_call_stack` which tries to
use the module frames but they were already freed by this line:
`DEINIT_VEC(store->modules, wasm_module_vec_delete)`
This PR swaps the order that these are deleted so module is deleted after the instances.
Co-authored-by: Andrew Chambers <ncham@amazon.com>
Use pre-created exec_env for instantiation and module_malloc/free,
use the same exec_env of the current thread to avoid potential
unexpected behavior.
And remove unnecessary shared_mem_lock in wasm_module_free,
which may cause dead lock.
Use the shared memory's shared_mem_lock to lock the whole atomic.wait and
atomic.notify processes, and use it for os_cond_reltimedwait and os_cond_notify,
so as to make the whole processes actual atomic operations:
the original implementation accesses the wait address with shared_mem_lock
and uses wait_node->wait_lock for os_cond_reltimedwait, which is not an atomic
operation.
And remove the unnecessary wait_map_lock and wait_lock, since the whole
processes are already locked by shared_mem_lock.
- Remove notify_stale_threads_on_exception and change atomic.wait
to be interruptible by keep waiting and checking every one second,
like the implementation of poll_oneoff in libc-wasi
- Wait all other threads exit and then get wasi exit_code to avoid
getting invalid value
- Inherit suspend_flags of parent thread while creating new thread to
avoid terminated flag isn't set for new thread
- Fix wasi-threads test case update_shared_data_and_alloc_heap
- Add "Lib wasi-threads enabled" prompt for cmake
- Fix aot get exception, use aot_copy_exception instead
Fix a data race for test main_proc_exit_wait.c from #1963.
And fix atomic_wait logic that was wrong before:
- a thread 1 started executing wasm instruction wasm_atomic_wait
but hasn't reached waiting on condition variable
- a main thread calls proc_exit and notifies all the threads that reached
waiting on condition variable
Which leads to thread 1 hang on waiting on condition variable after that
Now it's atomically checked whether proc_exit was already called.
- Implement atomic.fence to ensure a proper memory synchronization order
- Destroy exec_env_singleton first in wasm/aot deinstantiation
- Change terminate other threads to wait for other threads in
wasm_exec_env_destroy
- Fix detach thread in thread_manager_start_routine
- Fix duplicated lock cluster->lock in wasm_cluster_cancel_thread
- Add lib-pthread and lib-wasi-threads compilation to Windows CI
Raising "wasi proc exit" exception, spreading it to other threads and then
clearing it in all threads may result in unexpected behavior: the sub thread
may end first, handle the "wasi proc exit" exception and clear exceptions
of other threads, including the main thread. And when main thread's
exception is cleared, it may continue to run and throw "unreachable"
exception. This also leads to some assertion failed.
Ignore exception spreading for "wasi proc exit" and don't clear exception
of other threads to resolve the issue.
And add suspend flag check after atomic wait since the atomic wait may
be notified by other thread when exception occurs.
Add shared memory lock when accessing the address to atomic wait/notify
inside linear memory to resolve its data race issue.
And statically initialize the goto table of interpreter labels to resolve the
data race issue of accessing the table.
The function has been there for long. While what it does look a bit unsafe
as it calls a function which may be not wasm-wise exported explicitly, it's
useful and widely used when implementing callback-taking APIs, including
our pthread_create's implementation.
Multiple threads generated from the same module should use the same
lock to protect the atomic operations.
Before this PR, each thread used a different lock to protect atomic
operations (e.g. atomic add), making the lock ineffective.
Fix#1958.
Add APIs to help prepare the imports for the wasm-c-api `wasm_instance_new`:
- wasm_importtype_is_linked
- wasm_runtime_is_import_func_linked
- wasm_runtime_is_import_global_linked
- wasm_extern_new_empty
For wasm-c-api, developer may use `wasm_module_imports` to get the import
types info, check whether an import func/global is linked with the above API,
and ignore the linking of an import func/global with `wasm_extern_new_empty`.
Sample `wasm-c-api-import` is added and document is updated.
Enable setting running mode when executing a wasm bytecode file
- Four running modes are supported: interpreter, fast-jit, llvm-jit and multi-tier-jit
- Add APIs to set/get the default running mode of the runtime
- Add APIs to set/get the running mode of a wasm module instance
- Add running mode options for iwasm command line tool
And add size/opt level options for LLVM JIT
The definitions `enum WASMExceptionID` in the compilation of wamrc and the compilation
of Fast JIT are different, since the latter enables the Fast JIT macro while the former doesn't.
This causes that the exception ID in AOT file generated by wamrc may be different from
iwasm binary compiled with Fast JIT enabled, and may result in unexpected behavior.
Remove the macro control to resolve it.
This syscall doesn't need allocating stack or TLS and it's expected from the application
to do that instead. E.g. WASI-libc already does this for `pthread_create`.
Also fix some of the examples to allocate memory for stack and not use stack before
the stack pointer is set to a correct value.
Support modes:
- run a commander module only
- run a reactor module only
- run a commander module and a/multiple reactor modules together
commander propagates WASIArguments to reactors
Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
When a wasm module is duplicated instantiated with wasm_instance_new,
the function import info of the previous instantiation may be overwritten by
the later instantiation, which may cause unexpected behavior.
Store the function import info into the module instance to fix the issue.
Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
For now this implementation uses thread manager.
Not sure whether thread manager is needed in that case. In the future there'll be likely another syscall added (for pthread_exit) and for that we might need some kind of thread management - with that in mind, we keep thread manager for now and will refactor this later if needed.
When a wasm module is duplicated instantiated with wasm_instance_new,
the function import info of the previous instantiation may be overwritten by
the later instantiation, which may cause unexpected behavior.
Store the function import info into the module instance to fix the issue.
