Fix the warnings and issues reported:
- in Windows platform
- by CodeQL static code analyzing
- by Coverity static code analyzing
And update CodeQL script to build exception handling and memory features.
Adding a new cmake flag (cache variable) `WAMR_BUILD_MEMORY64` to enable
the memory64 feature, it can only be enabled on the 64-bit platform/target and
can only use software boundary check. And when it is enabled, it can support both
i32 and i64 linear memory types. The main modifications are:
- wasm loader & mini-loader: loading and bytecode validating process
- wasm runtime: memory instantiating process
- classic-interpreter: wasm code executing process
- Support memory64 memory in related runtime APIs
- Modify main function type check when it's memory64 wasm file
- Modify `wasm_runtime_invoke_native` and `wasm_runtime_invoke_native_raw` to
handle registered native function pointer argument when memory64 is enabled
- memory64 classic-interpreter spec test in `test_wamr.sh` and in CI
Currently, it supports memory64 memory wasm file that uses core spec
(including bulk memory proposal) opcodes and threads opcodes.
ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091https://github.com/bytecodealliance/wasm-micro-runtime/pull/3240https://github.com/bytecodealliance/wasm-micro-runtime/pull/3260
Implement the GC (Garbage Collection) feature for interpreter mode,
AOT mode and LLVM-JIT mode, and support most features of the latest
spec proposal, and also enable the stringref feature.
Use `cmake -DWAMR_BUILD_GC=1/0` to enable/disable the feature,
and `wamrc --enable-gc` to generate the AOT file with GC supported.
And update the AOT file version from 2 to 3 since there are many AOT
ABI breaks, including the changes of AOT file format, the changes of
AOT module/memory instance layouts, the AOT runtime APIs for the
AOT code to invoke and so on.
This PR adds the initial support for WASM exception handling:
* Inside the classic interpreter only:
* Initial handling of Tags
* Initial handling of Exceptions based on W3C Exception Proposal
* Import and Export of Exceptions and Tags
* Add `cmake -DWAMR_BUILD_EXCE_HANDLING=1/0` option to enable/disable
the feature, and by default it is disabled
* Update the wamr-test-suites scripts to test the feature
* Additional CI/CD changes to validate the exception spec proposal cases
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1884587513f3c68bebfe9ad759bccdfed8
Signed-off-by: Ricardo Aguilar <ricardoaguilar@siemens.com>
Co-authored-by: Chris Woods <chris.woods@siemens.com>
Co-authored-by: Rene Ermler <rene.ermler@siemens.com>
Co-authored-by: Trenner Thomas <trenner.thomas@siemens.com>
In some scenarios there may be lots of callings to AOT/JIT functions from the
host embedder, which expects good performance for the calling process, while
in the current implementation, runtime calls the wasm_runtime_invoke_native
to prepare the array of registers and stacks for the invokeNative assemble code,
and the latter then puts the elements in the array to physical registers and
native stacks and calls the AOT/JIT function, there may be many data copying
and handlings which impact the performance.
This PR registers some quick AOT/JIT entries for some simple wasm signatures,
and let runtime call the entry to directly invoke the AOT/JIT function instead of
calling wasm_runtime_invoke_native, which speedups the calling process.
We may extend the mechanism next to allow the developer to register his quick
AOT/JIT entries to speedup the calling process of invoking the AOT/JIT functions
for some specific signatures.
Currently, `data.drop` instruction is implemented by directly modifying the
underlying module. It breaks use cases where you have multiple instances
sharing a single loaded module. `elem.drop` has the same problem too.
This PR fixes the issue by keeping track of which data/elem segments have
been dropped by using bitmaps for each module instances separately, and
add a sample to demonstrate the issue and make the CI run it.
Also add a missing check of dropped elements to the fast-jit `table.init`.
Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/2735
Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/2772
To allow non-POSIX platforms such as Windows to support WASI libc
filesystem functionality, create a set of wrapper functions which provide a
platform-agnostic interface to interact with the host filesystem. For now,
the Windows implementation is stubbed but this will be implemented
properly in a future PR. There are no functional changes in this change,
just a reorganization of code to move any direct POSIX references out of
posix.c in the libc implementation into posix_file.c under the shared
POSIX sources.
See https://github.com/bytecodealliance/wasm-micro-runtime/issues/2495 for a
more detailed overview of the plan to port the WASI libc filesystem to Windows.
Multiple threads generated from the same module should use the same
lock to protect the atomic operations.
Before this PR, each thread used a different lock to protect atomic
operations (e.g. atomic add), making the lock ineffective.
Fix#1958.
Enable setting running mode when executing a wasm bytecode file
- Four running modes are supported: interpreter, fast-jit, llvm-jit and multi-tier-jit
- Add APIs to set/get the default running mode of the runtime
- Add APIs to set/get the running mode of a wasm module instance
- Add running mode options for iwasm command line tool
And add size/opt level options for LLVM JIT
Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
When a wasm module is duplicated instantiated with wasm_instance_new,
the function import info of the previous instantiation may be overwritten by
the later instantiation, which may cause unexpected behavior.
Store the function import info into the module instance to fix the issue.
Refine the generated LLVM IRs at the beginning of each LLVM AOT/JIT function
to fasten the LLVM IR optimization:
- Only create argv_buf if there are func calls in this function
- Only create native stack bound if stack bound check is enabled
- Only create aux stack info if there is opcode set_global_aux_stack
- Only create native symbol if indirect_mode is enabled
- Only create memory info if there are memory operations
- Only create func_type_indexes if there is opcode call_indirect
Refactor LLVM JIT for some purposes:
- To simplify the source code of JIT compilation
- To simplify the JIT modes
- To align with LLVM latest changes
- To prepare for the Multi-tier JIT compilation, refer to #1302
The changes mainly include:
- Remove the MCJIT mode, replace it with ORC JIT eager mode
- Remove the LLVM legacy pass manager (only keep the LLVM new pass manager)
- Change the lazy mode's LLVM module/function binding:
change each function in an individual LLVM module into all functions in a single LLVM module
- Upgraded ORC JIT to ORCv2 JIT to enable lazy compilation
Refer to #1468
Refactor the layout of interpreter and AOT module instance:
- Unify the interp/AOT module instance, use the same WASMModuleInstance/
WASMMemoryInstance/WASMTableInstance data structures for both interpreter
and AOT
- Make the offset of most fields the same in module instance for both interpreter
and AOT, append memory instance structure, global data and table instances to
the end of module instance for interpreter mode (like AOT mode)
- For extra fields in WASM module instance, use WASMModuleInstanceExtra to
create a field `e` for interpreter
- Change the LLVM JIT module instance creating process, LLVM JIT uses the WASM
module and module instance same as interpreter/Fast-JIT mode. So that Fast JIT
and LLVM JIT can access the same data structures, and make it possible to
implement the Multi-tier JIT (tier-up from Fast JIT to LLVM JIT) in the future
- Unify some APIs: merge some APIs for module instance and memory instance's
related operations (only implement one copy)
Note that the AOT ABI is same, the AOT file format, AOT relocation types, how AOT
code accesses the AOT module instance and so on are kept unchanged.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1384
Implement more socket APIs, refer to #1336 and below PRs:
- Implement wasi_addr_resolve function (#1319)
- Fix socket-api byte order issue when host/network order are the same (#1327)
- Enhance sock_addr_local syscall (#1320)
- Implement sock_addr_remote syscall (#1360)
- Add support for IPv6 in WAMR (#1411)
- Implement ns lookup allowlist (#1420)
- Implement sock_send_to and sock_recv_from system calls (#1457)
- Added http downloader and multicast socket options (#1467)
- Fix `bind()` calls to receive the correct size of `sockaddr` structure (#1490)
- Assert on correct parameters (#1505)
- Copy only received bytes from socket recv buffer into the app buffer (#1497)
Co-authored-by: Marcin Kolny <mkolny@amazon.com>
Co-authored-by: Marcin Kolny <marcin.kolny@gmail.com>
Co-authored-by: Callum Macmillan <callumimacmillan@gmail.com>
Normalize wasm types, for the two wasm types, if their parameter types
and result types are the same, we only save one copy, so as to reduce
the footprint and simplify the type comparison in opcode CALL_INDIRECT.
And fix issue in interpreter globals_instantiate, and remove used codes.
Import WAMR Fast JIT which is a lightweight JIT with quick startup, small footprint,
relatively good performance (~40% to ~50% of LLVM JIT) and good portability.
Platforms supported: Linux, MacOS and Linux SGX.
Arch supported: x86-64.
Refine opcode br_table for classic interpreter as there may be a lot of
leb128 decoding when the br count is big:
1. Use the bytecode itself to store the decoded leb br depths if each
decoded depth can be stored with one byte
2. Create br_table cache to store the decode leb br depths if the decoded
depth cannot be stored with one byte
After the optimization, the class interpreter can access the br depths array
with index, no need to decode the leb128 again.
And fix function record_fast_op() return value unchecked issue in source
debugging feature.
Refer to [Networking API design](https://github.com/WebAssembly/WASI/issues/370)
and [feat(socket): berkeley socket API v2](https://github.com/WebAssembly/WASI/pull/459):
- Support the socket API of synchronous mode, including `socket/bind/listen/accept/send/recv/close/shutdown`,
the asynchronous mode isn't supported yet.
- Support adding `--addr-pool=<pool1,pool2,..>` argument for command line to identify the valid ip address range
- Add socket-api sample and update the document
Currently when calling wasm_runtime_call_wasm() to invoke wasm function
with externref type argument from runtime embedder, developer needs to
use wasm_externref_obj2ref() to convert externref obj into an internal ref
index firstly, which is not convenient to developer.
To align with GC feature in which all the references passed to
wasm_runtime_call_wasm() can be object pointers directly, we change the
interface of wasm_runtime_call_wasm() to allow to pass object pointer
directly for the externref argument, and refactor the related codes, update
the related samples and the document.
Refer to https://github.com/WebAssembly/WASI/blob/main/design/application-abi.md
to check the WASI ABI compatibility:
- Command (main module) may export _start function with signature "()"
- Reactor (sub module) may export _initialize function with signature "()"
- _start and _initialize can not be exported at the same time
- Reactor cannot export _start function
- Command and Reactor must export memory
And
- Rename module->is_wasi_module to module->import_wasi_api
- Refactor wasm_loader_find_export()
- Remove MULTI_MODULE related codes from mini_loader
- Update multi-module samples
- Fix a "use-after-free" issue. Since we reuse the memory instance of sub module,
just to protect it from freeing an imported memory instance
Refine some codes in wasm loader
Add -Wshadow to gcc compile flags and fix some variable shadowed issues
Fix function parameter/return types not checked issue
Fix fast-interp loader reserve_block_ret() not handle V128 return type issue
Fix mini loader load_table_segment_section() failed issue
Add detailed comments for argc argument in wasm_runtime_call_wasm()
- move the wait_cond from exec_env to debug_instance, so the debug thread can be waken up by any threads
- process more general query message from debugger
- refine debug instance create/destroy mechanism
- avoid creating debug instance during module instantiating
- avoid blocking execution thread during creating debug instance
- update related documents
Enable emitting custom name section to aot file when adding
`--enable-dump-call-stack` or `--enable-dump-call-stack` to
wamrc and there is custom name section in wasm file, which
can be generated by wasi-sdk/emcc "-g" option. So aot runtime
can also get the function name from the custom name section
instead of export section, to which developer should use
`--export-all` for wasi-sdk/emcc to generate export function
names.
Implement source debugging feature for classic interpreter and AOT:
- use `cmake -DWAMR_BUILD_DEBUG_INTERP=1` to enable interpreter debugging
- use `cmake -DWAMR_BUILD_DEBUG_AOT=1` to enable AOT debugging
See doc/source_debugging.md for more details.
Implement spec reference-types proposal for interpreter, AOT and JIT, update documents and add sample. And upgrade AOT_CURRENT_VERSION to 3 as AOT file format and AOT module instance layout are changed.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Re-implement aot call_indirect opcode translation: when calling non-import function, translate it by LLVM call IR to call the function in AOTed code, so as to avoid calling runtime aot_call_indirect API which is much slower. For import function, keep calling aot_call_indirect API due to the possible pointer/string argument conversion.
And add prompt info while app heap is corrupted, change emit_leb to emit_uint32 inter fast-interp to refine footprint.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Update wasm app build scripts for wasi-sdk-12.0: add --export=__main_argc_argv, remove --no-threads
Lookup function with name "__main_argc_argv" as main function besides "main"
Change module_malloc to runtime_malloc in wasi native lib
Refine classic interpreter op_block and op_br_table
Refine faster interpreter op_br_table
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Add the function name field for internal function struct
Signed-off-by: Zhongmin Wu <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Co-authored-by: Zhongmin Wu <vwzm@live.com>
