Commit Graph

238 Commits

Author SHA1 Message Date
Wenyong Huang
e1d0c27ef9
Fix ref.func forward-declared function check (#2099)
When ref.func opcode refers to a function whose function index no smaller than
current function, the destination func should be forward-declared: it is declared
in the table element segments, or is declared in the export list.
2023-04-03 15:55:24 +08:00
Wenyong Huang
b0736e2e88
Fix issues reported by Coverity (#2083)
Get exec_env_tls at the beginning of execute_post_instantiate_functions
to avoid it is uninitialized when is_sub_inst is false.
2023-03-29 19:40:52 +08:00
Wenyong Huang
10f1bf3af7
Fix module_malloc/module_free issues (#2072)
Try using existing exec_env to execute wasm app's malloc/free func and
execute post instantiation functions. Create a new exec_env only when
no existing exec_env was found.
2023-03-28 18:31:09 +08:00
Wenyong Huang
605c8b07dc
Fix issue of Multi-tier JIT (#2056) 2023-03-25 11:15:05 +08:00
Wenyong Huang
3977f0b22a
Use pre-created exec_env for instantiation and module_malloc/free (#2047)
Use pre-created exec_env for instantiation and module_malloc/free,
use the same exec_env of the current thread to avoid potential
unexpected behavior.

And remove unnecessary shared_mem_lock in wasm_module_free,
which may cause dead lock.
2023-03-23 19:19:47 +08:00
Wenyong Huang
49d439a3bc
Fix/Simplify the atomic.wait/nofity implementations (#2044)
Use the shared memory's shared_mem_lock to lock the whole atomic.wait and
atomic.notify processes, and use it for os_cond_reltimedwait and os_cond_notify,
so as to make the whole processes actual atomic operations:
the original implementation accesses the wait address with shared_mem_lock
and uses wait_node->wait_lock for os_cond_reltimedwait, which is not an atomic
operation.

And remove the unnecessary wait_map_lock and wait_lock, since the whole
processes are already locked by shared_mem_lock.
2023-03-23 09:21:16 +08:00
Xu Jun
d75cb3224f
Fix dead lock in source debugger (#2040) 2023-03-20 08:17:22 +08:00
Wenyong Huang
f279ba84ee
Fix multi-threading issues (#2013)
- Implement atomic.fence to ensure a proper memory synchronization order
- Destroy exec_env_singleton first in wasm/aot deinstantiation
- Change terminate other threads to wait for other threads in
  wasm_exec_env_destroy
- Fix detach thread in thread_manager_start_routine
- Fix duplicated lock cluster->lock in wasm_cluster_cancel_thread
- Add lib-pthread and lib-wasi-threads compilation to Windows CI
2023-03-08 10:57:22 +08:00
Enrico Loparco
e8d718096d
Add/reorganize locks for thread synchronization (#1995)
Attempt to fix data races when using threads.
- Protect access (from multiple threads) to exception and memory
- Fix shared memory lock usage
2023-03-04 08:15:26 +08:00
Enrico Loparco
52e26e59cf
Add lock to protect the operations of accessing exec env (#1991)
Data race may occur when accessing exec_env's fields, e.g. suspend_flags
and handle. Add lock `exec_env->wait_lock` for them to resolve the issue.
2023-02-27 19:53:41 +08:00
Wenyong Huang
38c67b3f48
thread-mgr: Fix spread "wasi proc exit" exception and atomic.wait issues (#1988)
Raising "wasi proc exit" exception, spreading it to other threads and then
clearing it in all threads may result in unexpected behavior: the sub thread
may end first, handle the "wasi proc exit" exception and clear exceptions
of other threads, including the main thread. And when main thread's
exception is cleared, it may continue to run and throw "unreachable"
exception. This also leads to some assertion failed.

Ignore exception spreading for "wasi proc exit" and don't clear exception
of other threads to resolve the issue.

And add suspend flag check after atomic wait since the atomic wait may
be notified by other thread when exception occurs.
2023-02-24 20:05:39 +08:00
Wenyong Huang
d2772c4153
Re-org calling post instantiation functions (#1972)
- Use execute_post_instantiate_functions to call start, _initialize,
  __post_instantiate, __wasm_call_ctors functions after instantiation
- Always call start function for both main instance and sub instance
- Only call _initialize and __post_instantiate for main instance
- Only call ___wasm_call_ctors for main instance and when bulk memory
  is enabled and wasi import functions are not found
- When hw bound check is enabled, use the existing exec_env_tls
  to call func for sub instance, and switch exec_env_tls's module inst
  to current module inst to avoid checking failure and using the wrong
  module inst
2023-02-22 12:24:11 +08:00
Enrico Loparco
bc60064de5
Fix data races in atomic wait/notify and interp goto table (#1971)
Add shared memory lock when accessing the address to atomic wait/notify
inside linear memory to resolve its data race issue.

And statically initialize the goto table of interpreter labels to resolve the
data race issue of accessing the table.
2023-02-22 11:43:49 +08:00
Wenyong Huang
739acfc908
Merge pull request #1966 from bytecodealliance/dev/wasi_threads
Merge wasi_threads to main
2023-02-17 17:42:47 +08:00
Wenyong Huang
ef3a683392
Don't call start/initialize in child thread's instantiation (#1967)
The start/initialize functions of wasi module are to do some initialization work
during instantiation, which should be only called one time in the instantiation
of main instance. For example, they may initialize the data in linear memory,
if the data is changed later by the main instance, and re-initialized again by
the child instance, unexpected behaviors may occur.

And clear a shadow warning in classic interpreter.
2023-02-17 15:11:05 +08:00
Wenyong Huang
e170c355a2 Merge branch main into dev/wasi_threads 2023-02-17 08:46:12 +08:00
Enrico Loparco
216dc43ab4
Use shared memory lock for threads generated from same module (#1960)
Multiple threads generated from the same module should use the same
lock to protect the atomic operations.

Before this PR, each thread used a different lock to protect atomic
operations (e.g. atomic add), making the lock ineffective.

Fix #1958.
2023-02-16 11:54:19 +08:00
liang.he
3698f2279b
Improve wasm-c-api instantiation-time linking (#1902)
Add APIs to help prepare the imports for the wasm-c-api `wasm_instance_new`:
- wasm_importtype_is_linked
- wasm_runtime_is_import_func_linked
- wasm_runtime_is_import_global_linked
- wasm_extern_new_empty

For wasm-c-api, developer may use `wasm_module_imports` to get the import
types info, check whether an import func/global is linked with the above API,
and ignore the linking of an import func/global with `wasm_extern_new_empty`.

Sample `wasm-c-api-import` is added and document is updated.
2023-02-13 15:06:04 +08:00
Wenyong Huang
c3e9b66b2a
Fix jit memory overwritten after instance deinstantiate (#1936)
When de-instantiating the wasm module instance, remove it from the module's
instance list before freeing func_ptrs and fast_jit_func_ptrs of the instance, to avoid
accessing these freed memory in the JIT backend compilation threads.
2023-02-03 11:15:03 +08:00
Wenyong Huang
40a14b51c5
Enable running mode control for runtime and module instance (#1923)
Enable setting running mode when executing a wasm bytecode file
- Four running modes are supported: interpreter, fast-jit, llvm-jit and multi-tier-jit
- Add APIs to set/get the default running mode of the runtime
- Add APIs to set/get the running mode of a wasm module instance
- Add running mode options for iwasm command line tool

And add size/opt level options for LLVM JIT
2023-02-02 18:16:01 +08:00
Wenyong Huang
7bb78dc260
Remove fast jit macro in enum WASMExceptionID (#1933)
The definitions `enum WASMExceptionID` in the compilation of wamrc and the compilation
of Fast JIT are different, since the latter enables the Fast JIT macro while the former doesn't.
This causes that the exception ID in AOT file generated by wamrc may be different from
iwasm binary compiled with Fast JIT enabled, and may result in unexpected behavior.
Remove the macro control to resolve it.
2023-02-02 15:23:44 +08:00
Wenyong Huang
edee2eb956
Loosen the wasi abi compatibility check in loader (#1932)
Change an error to warning when checking wasi abi compatibility in loader, for rust case below:
#[no_mangle]
pub extern "C" fn main() {
    println!("foo");
}
compile it with `cargo build --target wasm32-wasi`, a wasm file is generated with wasi apis imported
and a "void main(void)" function exported.

Other runtime e.g. wasmtime allows to load it and execute the main function with `--invoke` option.
2023-02-02 13:06:30 +08:00
YAMAMOTO Takashi
7d3b2a8773
Make memory profiling show native stack usage (#1917) 2023-02-01 11:52:15 +08:00
Enrico Loparco
2eed50b303
Document how to use WASI threads in AOT mode (#1905)
Describe how to use WASI threads in AOT mode, following the discussion below:
  https://github.com/bytecodealliance/wasm-micro-runtime/pull/1867#discussion_r1070268062

Make aux stack boundary checks of wasi-threads always successful by setting
`exec_env->aux_stack_bottom` to UINT32_MAX and `exec_env->aux_stack_boundary` to 0
2023-01-27 18:32:33 +08:00
tonibofarull
9eed6686df
Refactor WASI-NN to simplify the support for multiple frameworks (#1834)
- Reorganize the library structure
- Use the latest version of `wasi-nn` wit (Oct 25, 2022):
    0f77c48ec1/wasi-nn.wit.md
- Split logic that converts WASM structs to native structs in a separate file
- Simplify addition of new frameworks
2023-01-25 18:32:40 +08:00
Wenyong Huang
c7141894fb Merge branch main into dev/wasi_threads 2023-01-21 13:18:55 +08:00
Xu Jun
cadf9d0ad3
Main thread spread exception when thread-mgr is enabled (#1889)
And refactor clear_wasi_proc_exit_exception, refer to
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1869
2023-01-20 08:54:27 +08:00
Xu Jun
e696ac36d7
Fix potential block issue in source debugger (#1887)
Fix issue reported in #1860
2023-01-17 08:45:29 +08:00
YAMAMOTO Takashi
5f0f8fe7ee
Relax wasi memory export check when wasi-threads is enabled (#1893)
https://github.com/WebAssembly/wasi-threads/issues/22
https://github.com/WebAssembly/WASI/issues/502
2023-01-16 20:21:51 +08:00
Martin Klang
622cdbefd6
Prevent undefined behavior from c_api_func_imports == NULL (#1883)
The module instance's c_api_func_imports may be NULL under some circumstances,
add checks before accessing it.
2023-01-14 07:52:39 +08:00
Marcin Kolny
0e2382a959
Disable aux stack allocations for threads spawned by wasi_thread_start (#1867)
This syscall doesn't need allocating stack or TLS and it's expected from the application
to do that instead. E.g. WASI-libc already does this for `pthread_create`.

Also fix some of the examples to allocate memory for stack and not use stack before
the stack pointer is set to a correct value.
2023-01-09 20:36:34 +08:00
Wenyong Huang
1f4580fbd8
Enable CI wasi test suite for x86-32 classic/fast interpreter (#1866)
The original CI didn't actually run wasi test suite for x86-32 since the `TEST_ON_X86_32=true`
isn't written into $GITHUB_ENV.

And refine the error output when failed to link import global.
2023-01-06 17:31:39 +08:00
liang.he
7401718311
Report error in instantiation when meeting unlinked import globals (#1859) 2023-01-06 15:24:11 +08:00
Wenyong Huang
3403f0a7f4 Merge main into dev/wasi_threads 2023-01-03 18:15:23 +08:00
Wenyong Huang
0090d3e3fc
Fix issue of resolving func name in custom name section (#1849)
Should use import_function_count but not import_count to calculate
the func_index in handle_name_section when custom name section
feature is enabled.

And clear the compile warnings of mini loader.
2022-12-30 14:37:04 +08:00
liang.he
676c3c7b04
Fix failure about preopen of reactor modules (#1816)
Support modes:
- run a commander module only
- run a reactor module only
- run a commander module and a/multiple reactor modules together
  commander propagates WASIArguments to reactors
2022-12-27 12:59:17 +08:00
Wenyong Huang
e8ce4c542e Implement Multi-tier JIT (#1774)
Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.

Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
2022-12-20 14:37:07 +08:00
Callum Macmillan
4d0f5cfade Fix watchpoint segfault when using debug interp without server (#1806) 2022-12-20 14:37:07 +08:00
Wenyong Huang
8fa7f1c4cf Upgrade version number to 1.1.2 (#1788)
Upgrade version number to 1.1.2, update RELEASE_NOTES.md and
clear several compile warnings.
2022-12-20 14:37:07 +08:00
Callum Macmillan
ead6478285 Add memory watchpoint support for source debugger (#1762)
Allow to add watchpoints to variables for source debugging. For instance:
`breakpoint set variable var`
will pause WAMR execution when the address at var is written to.
Can also set read/write watchpoints by passing r/w flags. This will pause
execution when the address at var is read:
`watchpoint set variable -w read var`

Add two linked lists for read/write watchpoints. When the debug message
handler receives a watchpoint request, it adds/removes to one/both of these
lists. In the interpreter, when an address is read or stored to, check whether
the address is in these lists. If so, throw a sigtrap and suspend the process.
2022-12-20 14:37:07 +08:00
Wenyong Huang
37e9b9c510 Fix wasm-c-api import func link issue in wasm_instance_new (#1787)
When a wasm module is duplicated instantiated with wasm_instance_new,
the function import info of the previous instantiation may be overwritten by
the later instantiation, which may cause unexpected behavior.

Store the function import info into the module instance to fix the issue.
2022-12-20 14:37:07 +08:00
Wenyong Huang
14288f59b0
Implement Multi-tier JIT (#1774)
Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.

Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
2022-12-19 11:24:46 +08:00
Marcin Kolny
0f637cff07
Enable aux stack allocations on application heap (#1799)
This is necessary for WASI threads as the aux stack should be managed by the application.
See https://github.com/bytecodealliance/wasm-micro-runtime/issues/1790 for details.
2022-12-19 10:58:12 +08:00
Callum Macmillan
ca0b5cf816
Fix watchpoint segfault when using debug interp without server (#1806) 2022-12-13 15:16:43 +08:00
Wenyong Huang
0d9e527fa0
Upgrade version number to 1.1.2 (#1788)
Upgrade version number to 1.1.2, update RELEASE_NOTES.md and
clear several compile warnings.
2022-12-08 12:21:34 +08:00
Callum Macmillan
c3d66f916e
Add memory watchpoint support for source debugger (#1762)
Allow to add watchpoints to variables for source debugging. For instance:
`breakpoint set variable var`
will pause WAMR execution when the address at var is written to.
Can also set read/write watchpoints by passing r/w flags. This will pause
execution when the address at var is read:
`watchpoint set variable -w read var`

Add two linked lists for read/write watchpoints. When the debug message
handler receives a watchpoint request, it adds/removes to one/both of these
lists. In the interpreter, when an address is read or stored to, check whether
the address is in these lists. If so, throw a sigtrap and suspend the process.
2022-12-07 17:18:28 +08:00
Wenyong Huang
9d52960e4d
Fix wasm-c-api import func link issue in wasm_instance_new (#1787)
When a wasm module is duplicated instantiated with wasm_instance_new,
the function import info of the previous instantiation may be overwritten by
the later instantiation, which may cause unexpected behavior.

Store the function import info into the module instance to fix the issue.
2022-12-07 16:43:04 +08:00
Wenyong Huang
1652f22a77
Fix issues reported by Coverity (#1775)
Fix some issues reported by Coverity and fix windows exception
check with guard page issue
2022-12-01 19:24:13 +08:00
Wenyong Huang
ce3458da99
Refine AOT exception check when function return (#1752)
Refine AOT exception check in the caller when returning from callee function,
remove the exception check instructions when hw bound check is enabled to
improve the performance: create guard page to trigger signal handler when
exception occurs.
2022-11-30 20:18:28 +08:00
Huang Qi
93d3d09aa1
Clear some warnings and enable -Werror for NuttX (#1756) 2022-11-28 09:44:33 +08:00