Commit Graph

197 Commits

Author SHA1 Message Date
liang.he
b2eb7d838d
Update spec test to latest commit (#3293)
- Update spec test cases to commit bc76fd79cfe61033d7f4ad4a7e8fc4f996dc5ba8 on Apr. 3
- Update wabt binary to 1.0.34 to support newer spec cases
- Add comparison between table declared elem type and table elem segment value type
- Add a function to decide whether to execute test cases in a running mode
- Keep using interpreter in GC spec because wat2wasm in wabt can't compile if.wast w/o errors
- Re-factoring threads spec test case processing
- Since wabt 1.0.34 release isn't compatible with ubuntu 20.04, compile it from source code
- Disable CI to run aot multi-module temporarily, and will enable it in another PR
2024-05-17 10:40:47 +08:00
Enrico Loparco
6b1d81650d
Allow not copying the wasm binary in wasm-c-api and not referring to the binary in wasm/aot loader (#3389)
Add flag `LoadArgs.clone_wasm_binary` to control whether to clone the wasm/aot
binary in wasm-c-api module. If false, API `wasm_module_new_ex` won't clone the
binary, which may reduce the footprint.

Add flag `LoadArgs.wasm_binary_freeable` to control whether the wasm/aot binary
may be freed after instantiation for wamr API `wasm_runtime_load_ex`, if yes, then
for some running modes, the wasm/aot module doesn't refer to the input binary
again so developer can free it after instantiation to reduce the footprint.

And add API `wasm_module_is_underlying_binary_freeable` and
`wasm_runtime_is_underlying_binary_freeable` to check whether the input binary
can be freed after instantiation for wasm-c-api and wamr api.

And add sample to illustrate it.
2024-05-17 09:00:08 +08:00
Wenyong Huang
49c9fa31da
Fix typo of WAMR_CONFIGUABLE_BOUNDS_CHECKS (#3424)
Change to WAMR_CONFIGURABLE_BOUNDS_CHECKS, and fix CodeQL compilation errors
which were introduced by PR #3406.

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3393#discussion_r1591810998
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/9055318553/job/24876266629
2024-05-14 14:33:09 +08:00
TianlongLiang
c6d42db598
Correct Exception Handling tag type when GC is enabled (#3413)
Use `WASMFuncType` to represent tag_type in `WASMTagImport` and
`WASMTag` so that the type definition is consistent no matter to GC is
enabled or not.

This PR fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/3409.
2024-05-13 16:42:01 +08:00
Marcin Kolny
fe5e7a9981
Implement Memory64 support for AOT (#3362)
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3266
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091
2024-05-13 11:03:38 +08:00
Benbuck Nason
c85bada2a9
Add wasm module global type information APIs (#3406)
Support getting global type from `wasm_runtime_get_import_type` and
`wasm_runtime_get_export_type`, and add two APIs:

```C
wasm_valkind_t
wasm_global_type_get_valkind(const wasm_global_type_t global_type);

bool
wasm_global_type_get_mutable(const wasm_global_type_t global_type);
```
2024-05-10 09:15:58 +08:00
Wenyong Huang
a6c0cb74c9
wasm loader: Fix handling if block without op else (#3404)
If there is no else branch, make a virtual else opcode for easier integrity
check and to copy the correct results to the block return address for
fast-interp mode: change if block from `if ... end` to `if ... else end`.

Reported in issue #3386, #3387, #3388.
2024-05-09 16:46:26 +08:00
Benbuck Nason
1c2a8fca4e
Fix some more spelling issues (#3393) 2024-05-08 09:30:29 +08:00
TianlongLiang
79ae910a7c
Fix loader and mini-loader select potiential error (#3374)
Fix the issue reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3361.
2024-04-29 12:27:55 +08:00
Wenyong Huang
e44465d259
Enhance wasm loader checks for opcode br_table (#3352)
Fix the integer overflow issue when checking target branch depth in opcode
br_table, and fix is_32bit_type not check VALUE_TYPE_ANY issue, which may
cause wasm_loader_push_frame_offset push extra unneeded offset.
2024-04-25 20:07:57 +08:00
Wenyong Huang
18d363029c
wasm loader: Fix checks for opcode ref.func and opcode else (#3340)
Fix wasm loader integrity checks for opcode ref.func and opcode else:
for opcode ref.func, the function must be an import, exported, or present in a
table elem segment or global initializer to be used as the operand to ref.func,
for opcode else, there must not be an else opcode previously.

Reported in #3336 and #3337.

And fix mini loader PUSH_MEM_OFFSET/POP_MEM_OFFSET macro
definitions due to the introducing of memory64 feature.
2024-04-22 14:44:45 +08:00
Marcin Kolny
fd7f738451
Allow executing malloc/free from native in memory64 mode (#3315) 2024-04-19 17:15:54 +08:00
Wenyong Huang
68bd30c6f9
Enhance GC subtyping checks (#3317)
Enhance the GC subtyping checks:
- Fix issues in the type equivalence check
- Enable the recursive type subtyping check
- Add a equivalence type flag in defined types of aot file, if there is an
  equivalence type before, just set it true and re-use the previous type
- Normalize the defined types for interpreter and AOT
- Enable spec test case type-equivalence.wast and type-subtyping.wast,
  and enable some commented cases
- Enable set WAMR_BUILD_SANITIZER from cmake variable
2024-04-18 12:32:01 +08:00
Wenyong Huang
1c690b7561
Add more checks in wasm loader (#3300)
In opcode f32.const, f64.const and memory.copy, check whether the buffer
to read is out of the range of wasm file before reading it.
2024-04-11 19:56:48 +08:00
liang.he
bcc2a2d2e1
Sync simd opcode definitions spec (#3290)
Remove undefined simd opcodes.
2024-04-09 16:54:42 +08:00
liang.he
4ef724bbff
Enhance wasm loading with LoadArgs and support module names (#3265)
- Add new API wasm_runtime_load_ex() in wasm_export.h
  and wasm_module_new_ex in wasm_c_api.h
- Put aot_create_perf_map() into a separated file aot_perf_map.c
- In perf.map, function names include user specified module name
- Enhance the script to help flamegraph generations
2024-04-07 15:04:35 +08:00
Wenyong Huang
2013f1f7d7
Fix warnings/issues reported in Windows and by CodeQL/Coverity (#3275)
Fix the warnings and issues reported:
- in Windows platform
- by CodeQL static code analyzing
- by Coverity static code analyzing

And update CodeQL script to build exception handling and memory features.
2024-04-07 11:57:31 +08:00
Wenyong Huang
a23fa9f86c
Implement memory64 for classic interpreter (#3266)
Adding a new cmake flag (cache variable) `WAMR_BUILD_MEMORY64` to enable
the memory64 feature, it can only be enabled on the 64-bit platform/target and
can only use software boundary check. And when it is enabled, it can support both
i32 and i64 linear memory types. The main modifications are:

- wasm loader & mini-loader: loading and bytecode validating process 
- wasm runtime: memory instantiating process
- classic-interpreter: wasm code executing process
- Support memory64 memory in related runtime APIs
- Modify main function type check when it's memory64 wasm file
- Modify `wasm_runtime_invoke_native` and `wasm_runtime_invoke_native_raw` to
  handle registered native function pointer argument when memory64 is enabled
- memory64 classic-interpreter spec test in `test_wamr.sh` and in CI

Currently, it supports memory64 memory wasm file that uses core spec
(including bulk memory proposal) opcodes and threads opcodes.

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3240
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3260
2024-04-02 15:22:07 +08:00
Wenyong Huang
6ee71000f9
Fix dynamic offset not updated in op_br for block with ret type (#3269)
The PR #3259 reverted PR #3192, it fixes #3210 but makes #3170 failed again.

The workaround is that we should update `ctx->dynamic_offset` only for opcode br
and should not update it for opcode br_if. This PR fixes both issue #3170 and #3210.
2024-04-01 15:40:20 +08:00
Wenyong Huang
f933d4c829
loader: Remove updating ctx->dynamic_offset in emit_br_info (#3259)
Should not update `ctx->dynamic_offset` in emit_br_info, since the `Part e` only
sets the dst offsets, the operand stack should not be changed, e.g., the stack
operands are to be used by the opcodes followed by `br_if` opcode.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3210.
2024-03-28 16:07:02 +08:00
liang.he
498eb5d54a
Append \0 to every name string in aot name section (#3249)
Since strings in .name section in .wasm is not c-style, need to append
a `\0` to each string in .name section in AOT file when emitting.
2024-03-26 14:27:20 +08:00
liang.he
d8d8f8ce04
Implement apis to set and get the name of a wasm module (#3254)
Add API wasm_runtime_set_module_name and wasm_runtime_get_module_name,
and by default, a module's name is "" if the set module name api isn't called.
2024-03-26 12:10:13 +08:00
Wenyong Huang
5e2011ca1d
Fix compilation errors on esp-idf platform (#3224)
The issue was reported in #3208.
2024-03-14 21:31:44 +08:00
Wenyong Huang
0ee5ffce85
Refactor APIs and data structures as preliminary work for Memory64 (#3209)
# Change the data type representing linear memory address from u32 to u64

## APIs signature changes
- (Export)wasm_runtime_module_malloc
  - wasm_module_malloc
    - wasm_module_malloc_internal
  - aot_module_malloc
    - aot_module_malloc_internal
- wasm_runtime_module_realloc
  - wasm_module_realloc
    - wasm_module_realloc_internal
  - aot_module_realloc
    - aot_module_realloc_internal
- (Export)wasm_runtime_module_free
  - wasm_module_free
    - wasm_module_free_internal
  - aot_module_malloc
    - aot_module_free_internal
- (Export)wasm_runtime_module_dup_data
  - wasm_module_dup_data
  - aot_module_dup_data
- (Export)wasm_runtime_validate_app_addr
- (Export)wasm_runtime_validate_app_str_addr
- (Export)wasm_runtime_validate_native_addr
- (Export)wasm_runtime_addr_app_to_native
- (Export)wasm_runtime_addr_native_to_app
- (Export)wasm_runtime_get_app_addr_range
- aot_set_aux_stack
- aot_get_aux_stack
- wasm_set_aux_stack
- wasm_get_aux_stack
- aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert
  and jit_check_app_addr_and_convert
- wasm_exec_env_set_aux_stack
- wasm_exec_env_get_aux_stack
- wasm_cluster_create_thread
- wasm_cluster_allocate_aux_stack
- wasm_cluster_free_aux_stack

## Data structure changes
- WASMModule and AOTModule
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMExecEnv
  - field aux_stack_boundary and aux_stack_bottom
- AOTCompData
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMMemoryInstance(AOTMemoryInstance)
  - field memory_data_size and change __padding to is_memory64
- WASMModuleInstMemConsumption
  - field total_size and memories_size
- WASMDebugExecutionMemory
  - field start_offset and current_pos
- WASMCluster
  - field stack_tops

## Components that are affected by the APIs and data structure changes
- libc-builtin
- libc-emcc
- libc-uvwasi
- libc-wasi
- Python and Go Language Embedding
- Interpreter Debug engine
- Multi-thread: lib-pthread, wasi-threads and thread manager
2024-03-12 11:38:50 +08:00
Xu Jun
21819fcff6
Fix dynamic offset in BR for block with return type (#3192)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3170.
2024-03-01 10:15:17 +08:00
Wenyong Huang
2349df1271
Fix ref.func opcode check when GC is enabled (#3181)
The current code assumes that the element type of table segment can
be `funcref` only, but when GC is enabled, the type can be `(ref func)`
also.

Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/3168.
2024-02-27 09:31:19 +08:00
Wenyong Huang
169e164815
Fix wasm loader handling opcode br_table (#3176)
Fix the errors reported in the sanitizer test of nightly run CI.
When the stack is in polymorphic state, the stack operands may be changed
after pop and push operations (e.g. stack is empty but pop op can succeed
in polymorphic, and the push op can push a new operand to stack), this may
impact the following checks to other target blocks of the br_table opcode.
2024-02-23 14:56:20 +08:00
Wenyong Huang
3a0e86454e
fast-interp: Fix GC opcode ref.as_non_null (#3156)
The issue was found in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3151.
2024-02-17 11:54:49 +08:00
YAMAMOTO Takashi
b0c54c8a86
Fix a ubsan complaint "applying zero offset to null pointer" (#3160)
Set a type's `result_ref_type_maps` only when `ref_type_map_count > 0`.
2024-02-17 10:11:10 +08:00
Xu Jun
12f834aebd
fast-interp: Fix frame_offset overflow issue (#3149)
The issue was found in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3137.
2024-02-08 11:32:17 +08:00
Wenyong Huang
16a4d71b34
Implement GC (Garbage Collection) feature for interpreter, AOT and LLVM-JIT (#3125)
Implement the GC (Garbage Collection) feature for interpreter mode,
AOT mode and LLVM-JIT mode, and support most features of the latest
spec proposal, and also enable the stringref feature.

Use `cmake -DWAMR_BUILD_GC=1/0` to enable/disable the feature,
and `wamrc --enable-gc` to generate the AOT file with GC supported.

And update the AOT file version from 2 to 3 since there are many AOT
ABI breaks, including the changes of AOT file format, the changes of
AOT module/memory instance layouts, the AOT runtime APIs for the
AOT code to invoke and so on.
2024-02-06 20:47:11 +08:00
Enrico Loparco
cfa90ca44f
Use logger for runtime error/debug prints (#3097)
Change runtime internal error/debug prints from using `os_printf()`
to using `LOG_ERROR()`/`LOG_DEBUG()`.
2024-02-06 13:02:54 +08:00
Xu Jun
d6d5072cc6
Fix locel.set in polymorphic stack (#3135)
The issue was reported in #3123.
2024-02-05 17:41:33 +08:00
Wenyong Huang
b8ff98c810
Fix possible integer overflow in loader target block check (#3133)
Checking with `loader_ctx->csp_num < depth + 1` has potential integer overflow
issue when depth is UINT_MAX, change to `loader_ctx->csp_num - 1 < depth`
instead.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3130.
2024-02-05 16:00:58 +08:00
Wenyong Huang
06df58f20e
Fix loader check_wasi_abi_compatibility (#3126)
Assume that wasi exported `_start` and `_initialize` functions can not
be an import function.

Fixes issue #3122.
2024-02-04 17:27:43 +08:00
Wenyong Huang
2eb60060d8
Fix read and validation of misc/simd/atomic sub opcodes (#3115)
The format of sub opcodes after misc, simd and atomic prefix is leb u32.

The issue was found in #2921.
2024-02-02 12:03:58 +08:00
YAMAMOTO Takashi
edc3643909
EH: Fix validation of delegate opcode (#3107)
cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/1884#issuecomment-1914000294
2024-02-01 12:32:06 +08:00
Xu Jun
40e51faca8
fast-interp: Fix block with parameter in polymorphic stack issue (#3112)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3061.
2024-02-01 11:55:29 +08:00
YAMAMOTO Takashi
3111a86cb3
EH: Fix delegate parsing in wasm_loader_find_block_addr (#3106) 2024-02-01 07:46:49 +08:00
Xu Jun
51e25ef01f
fast-interp: Fix frame_offset pop order (#3101)
The issue was reported in #3062.
2024-01-31 19:26:27 +08:00
Xu Jun
5a99866c01
fast-interp: Fix stack recovery for else branch (#3100)
The issue was reported in #3090.
2024-01-31 11:23:23 +08:00
Wenyong Huang
af318bac81
Implement Exception Handling for classic interpreter (#3096)
This PR adds the initial support for WASM exception handling:
* Inside the classic interpreter only:
  * Initial handling of Tags
  * Initial handling of Exceptions based on W3C Exception Proposal
  * Import and Export of Exceptions and Tags
* Add `cmake -DWAMR_BUILD_EXCE_HANDLING=1/0` option to enable/disable
  the feature, and by default it is disabled
* Update the wamr-test-suites scripts to test the feature
* Additional CI/CD changes to validate the exception spec proposal cases

Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1884
587513f3c6
8bebfe9ad7
59bccdfed8

Signed-off-by: Ricardo Aguilar <ricardoaguilar@siemens.com>
Co-authored-by: Chris Woods <chris.woods@siemens.com>
Co-authored-by: Rene Ermler <rene.ermler@siemens.com>
Co-authored-by: Trenner Thomas <trenner.thomas@siemens.com>
2024-01-31 08:27:17 +08:00
Wenyong Huang
9f64340529
Fix some issues reported by CodeQL (#3064)
Refer to https://github.com/bytecodealliance/wasm-micro-runtime/pull/2812
and https://github.com/bytecodealliance/wasm-micro-runtime/security/code-scanning?query=pr%3A2812+is%3Aopen
2024-01-23 21:38:30 +08:00
Xu Jun
1977ad23ef
fast-interp: Fix dynamic offset error issue in else branch (#3058)
Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3026.
2024-01-19 19:58:12 +08:00
Wenyong Huang
23d2e0627c
Clear compilation warning and dead code (#3002) 2024-01-13 08:56:39 +08:00
liang.he
9121db5671
Fix a bug when emit the custom name section to aot file (#2987)
The content in custom name section is changed after loaded since the strings
are adjusted with '\0' appended, the emitted AOT file then cannot be loaded.
The PR disables changing the content for AOT compiler to resolve it.

And disable emitting custom name section for `wamrc --enable-dump-call-stack`,
instead, use `wamrc --emit-custom-sections=name` to emit it.
2024-01-11 13:14:54 +08:00
Wenyong Huang
b21f17dd6d
Refine AOT/JIT code call wasm-c-api import process (#2982)
Allow to invoke the quick call entry wasm_runtime_quick_invoke_c_api_import to
call the wasm-c-api import functions to speedup the calling process, which reduces
the data copying.

Use `wamrc --invoke-c-api-import` to generate the optimized AOT code, and set
`jit_options->quick_invoke_c_api_import` true in wasm_engine_new when LLVM JIT
is enabled.
2024-01-10 18:37:02 +08:00
Wenyong Huang
7c7684819d
Register quick call entries to speedup the aot/jit func call process (#2978)
In some scenarios there may be lots of callings to AOT/JIT functions from the
host embedder, which expects good performance for the calling process, while
in the current implementation, runtime calls the wasm_runtime_invoke_native
to prepare the array of registers and stacks for the invokeNative assemble code,
and the latter then puts the elements in the array to physical registers and
native stacks and calls the AOT/JIT function, there may be many data copying
and handlings which impact the performance.

This PR registers some quick AOT/JIT entries for some simple wasm signatures,
and let runtime call the entry to directly invoke the AOT/JIT function instead of
calling wasm_runtime_invoke_native, which speedups the calling process.

We may extend the mechanism next to allow the developer to register his quick
AOT/JIT entries to speedup the calling process of invoking the AOT/JIT functions
for some specific signatures.
2024-01-10 16:44:09 +08:00
Xu Jun
f96257bade
Fix fast-interp polymorphic stack processing (#2974)
Fix issue #2951, #2952 and #2953.
2024-01-04 10:00:36 +08:00
Wenyong Huang
1ee4767d97
Fix ref.func function declared check in wasm loader (#2972)
The forward-declare function reference in ref.func can be declared in table element segments,
no matter whether the segment mode is passive, active or declarative.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/2944.
2024-01-03 11:43:03 +08:00