Commit Graph

1011 Commits

Author SHA1 Message Date
Wenyong Huang
902aa525d1
Fix windows relocation string parsing issue (#3333)
The AOT compiler emits string with '\0' now, some string parsing in aot loader
for Windows platform need to be updated.
2024-04-19 16:09:03 +08:00
dongsheng28849455
ba59e56e19
User defined memory allocator for different purposes (#3316)
Some issues are related with memory fragmentation, which may cause
the linear memory cannot be allocated. In WAMR, the memory managed
by the system is often trivial, but linear memory usually directly allocates
a large block and often remains unchanged for a long time. Their sensitivity
and contribution to fragmentation are different, which is suitable for
different allocation strategies. If we can control the linear memory's allocation,
do not make it from system heap, the overhead of heap management might
be avoided.

Add `mem_alloc_usage_t usage` as the first argument for user defined
malloc/realloc/free functions when `WAMR_BUILD_ALLOC_WITH_USAGE` cmake
variable is set as 1, and make passing `Alloc_For_LinearMemory` to the
argument when allocating the linear memory.
2024-04-18 19:40:57 +08:00
Wenyong Huang
68bd30c6f9
Enhance GC subtyping checks (#3317)
Enhance the GC subtyping checks:
- Fix issues in the type equivalence check
- Enable the recursive type subtyping check
- Add a equivalence type flag in defined types of aot file, if there is an
  equivalence type before, just set it true and re-use the previous type
- Normalize the defined types for interpreter and AOT
- Enable spec test case type-equivalence.wast and type-subtyping.wast,
  and enable some commented cases
- Enable set WAMR_BUILD_SANITIZER from cmake variable
2024-04-18 12:32:01 +08:00
liang.he
42199f163e
Log warning if growing table failed (#3310) 2024-04-15 18:29:54 +08:00
Wenyong Huang
1c690b7561
Add more checks in wasm loader (#3300)
In opcode f32.const, f64.const and memory.copy, check whether the buffer
to read is out of the range of wasm file before reading it.
2024-04-11 19:56:48 +08:00
TianlongLiang
4e634bed3f
Add necessary comments for doxygen (#3299)
- Add necessary comments for doxygen to generate API documentation
- Update README.md to add a link for End-user APIs documentation
2024-04-11 07:50:08 +08:00
liang.he
bcc2a2d2e1
Sync simd opcode definitions spec (#3290)
Remove undefined simd opcodes.
2024-04-09 16:54:42 +08:00
Wenyong Huang
ef3babc658
interp: Restore context from prev_frame after tail calling a native function (#3283)
The current frame was freed before tail calling to an import or native function
and the prev_frame was set as exec_env's cur_frame, so after the tail calling,
we should recover context from prev_frame but not current frame.

Found in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3279.
2024-04-08 09:23:20 +08:00
Wenyong Huang
b11dbcba0a
thread mgr: Free aux stack only when it was allocated (#3282)
When thread manager is enabled, the aux stack of exec_env may be allocated
by wasm_cluster_allocate_aux_stack or disabled by setting aux_stack_bottom
as UINTPTR_MAX directly. For the latter, no need to free it.

And fix an issue when paring `--gc-heap-size=n` argument for iwasm, and
fix a variable shadowed warning in fast-jit.
2024-04-08 09:06:49 +08:00
liang.he
4ef724bbff
Enhance wasm loading with LoadArgs and support module names (#3265)
- Add new API wasm_runtime_load_ex() in wasm_export.h
  and wasm_module_new_ex in wasm_c_api.h
- Put aot_create_perf_map() into a separated file aot_perf_map.c
- In perf.map, function names include user specified module name
- Enhance the script to help flamegraph generations
2024-04-07 15:04:35 +08:00
Wenyong Huang
2013f1f7d7
Fix warnings/issues reported in Windows and by CodeQL/Coverity (#3275)
Fix the warnings and issues reported:
- in Windows platform
- by CodeQL static code analyzing
- by Coverity static code analyzing

And update CodeQL script to build exception handling and memory features.
2024-04-07 11:57:31 +08:00
YAMAMOTO Takashi
53f0941ffa
Revert "lldb_function_to_function_dbi: A hack to avoid crashing on C++ methods (#3190)" (#3281)
This reverts commit 0e8d949440.

Because it doesn't make much sense anymore after we disabled debug info
processing on C++ functions in:
"aot debug: process lldb_function_to_function_dbi only for C".
2024-04-06 15:00:48 +08:00
YAMAMOTO Takashi
b4cab84e5f
aot debug: Process lldb_function_to_function_dbi only for C (#3278)
This is a workaroud for:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3187
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3163
2024-04-03 18:03:31 +08:00
YAMAMOTO Takashi
dae09c0e03
aot debug: Fix a NULL dereference (#3274)
It happens on eg. a C function taking a structure argument.
2024-04-03 17:10:46 +08:00
YAMAMOTO Takashi
4806e4e298
LLVM 19: Switch to debug records (#3272)
References:
    https://llvm.org/docs/RemoveDIsDebugInfo.html
    https://github.com/llvm/llvm-project/pull/86529
2024-04-02 16:00:43 +08:00
Wenyong Huang
a23fa9f86c
Implement memory64 for classic interpreter (#3266)
Adding a new cmake flag (cache variable) `WAMR_BUILD_MEMORY64` to enable
the memory64 feature, it can only be enabled on the 64-bit platform/target and
can only use software boundary check. And when it is enabled, it can support both
i32 and i64 linear memory types. The main modifications are:

- wasm loader & mini-loader: loading and bytecode validating process 
- wasm runtime: memory instantiating process
- classic-interpreter: wasm code executing process
- Support memory64 memory in related runtime APIs
- Modify main function type check when it's memory64 wasm file
- Modify `wasm_runtime_invoke_native` and `wasm_runtime_invoke_native_raw` to
  handle registered native function pointer argument when memory64 is enabled
- memory64 classic-interpreter spec test in `test_wamr.sh` and in CI

Currently, it supports memory64 memory wasm file that uses core spec
(including bulk memory proposal) opcodes and threads opcodes.

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3240
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3260
2024-04-02 15:22:07 +08:00
YAMAMOTO Takashi
6b0b5de1c5
aot debug: Fix a few NULL dereferences on errors (#3273) 2024-04-02 14:30:08 +08:00
dongsheng28849455
250829c0cc
Set is_vram_word_align as false in aot_const_str_set_insert function (#3271)
Set `is_vram_word_align` as false in aot_const_str_set_insert function when
`const char *name` is not from vram required word_align reading.
2024-04-02 14:17:30 +08:00
Wenyong Huang
6ee71000f9
Fix dynamic offset not updated in op_br for block with ret type (#3269)
The PR #3259 reverted PR #3192, it fixes #3210 but makes #3170 failed again.

The workaround is that we should update `ctx->dynamic_offset` only for opcode br
and should not update it for opcode br_if. This PR fixes both issue #3170 and #3210.
2024-04-01 15:40:20 +08:00
Wenyong Huang
ec15b6bbad
Add lock and ref_count for runtime init (#3263)
Some environment may call wasm_runtime_full_init/wasm_runtime_init multiple
times without knowing that runtime is initialized or not, it is better to add lock
and increase reference count during initialization.

ps. https://github.com/bytecodealliance/wasm-micro-runtime/discussions/3253.
2024-04-01 12:39:46 +08:00
Wenyong Huang
9c8551cf75
Add cmake flag to control aot intrinsics (#3261)
Add cmake variable `-DWAMR_BUILD_AOT_INTRINSICS=1/0` to enable/disable
the aot intrinsic functions, which are normally used by AOT XIP feature, and
can be disabled to reduce the aot runtime binary size.

And refactor the code in aot_intrinsics.h/.c.
2024-04-01 11:26:05 +08:00
Wenyong Huang
f933d4c829
loader: Remove updating ctx->dynamic_offset in emit_br_info (#3259)
Should not update `ctx->dynamic_offset` in emit_br_info, since the `Part e` only
sets the dst offsets, the operand stack should not be changed, e.g., the stack
operands are to be used by the opcodes followed by `br_if` opcode.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3210.
2024-03-28 16:07:02 +08:00
liang.he
498eb5d54a
Append \0 to every name string in aot name section (#3249)
Since strings in .name section in .wasm is not c-style, need to append
a `\0` to each string in .name section in AOT file when emitting.
2024-03-26 14:27:20 +08:00
liang.he
d8d8f8ce04
Implement apis to set and get the name of a wasm module (#3254)
Add API wasm_runtime_set_module_name and wasm_runtime_get_module_name,
and by default, a module's name is "" if the set module name api isn't called.
2024-03-26 12:10:13 +08:00
Xu Jinyang
cef88deedb
Add wasi_ephemeral_nn module support (#3241)
Add `wasi_ephemeral_nn` module support with optional cmake variable,
which was mentioned in #3229.
2024-03-21 21:05:34 +08:00
Liangyu Zhang
b11a1d157d
GC: Add wasm_struct_obj_get_field_count API (#3236) 2024-03-18 16:26:30 +08:00
Wenyong Huang
ff296c1a62
Fix aot relocation symbols not found on windows 32-bit (#3231)
The symbols in windows 32-bit may start with '_' and can not be found
when resolving the relocations to them. This PR ignores the underscore
when handling the relocation name of AOT_FUNC_INTERNAL_PREFIX, and
redirect the relocation with name "_aot_stack_sizes" to the relocation with
name ".aot_stack_sizes" (the name of the data section created).

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3216
2024-03-18 09:51:38 +08:00
Wenyong Huang
5e2011ca1d
Fix compilation errors on esp-idf platform (#3224)
The issue was reported in #3208.
2024-03-14 21:31:44 +08:00
TianlongLiang
de803b2beb
Small refactor on WASMModuleInstance and fix Go/Python language bindings (#3227)
- Merge unused field `used_to_be_wasi_ctx` in `AOTModuleInstance` into `reserved` area
- Add field `memory_lock` in `WASMMemoryInstance` for future refactor
- Go binding: fix type error
    https://github.com/bytecodealliance/wasm-micro-runtime/issues/3220
- Python binding:
    type annotation uses the union operator "|", which requires Python version >=3.10
2024-03-14 15:30:28 +08:00
TianlongLiang
c3e33a96ea
Remove unused argument in wasm_runtime_lookup_function and refactor WASMModuleInstance (#3218)
Remove the unused parameter `signature` from `wasm_runtime_lookup_function`.

Refactor the layout of WASMModuleInstance structure:
- move common data members `c_api_func_imports` and `cur_exec_env` from
  `WASMModuleInstanceExtraCommon` to `WASMModuleInstance`
- In `WASMModuleInstance`, enlarge `reserved[3]` to `reserved[5]` in case that
  we need to add more fields in the future

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/2530
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3202
2024-03-13 12:28:45 +08:00
Marcin Kolny
ce44e0ec0c
Allow converting the zero wasm address to native (#3215)
This allows to know the beginning of the wasm address space. At the moment
to achieve that, we need to apply a `hack wasm_runtime_addr_app_to_native(X)-X`
to get the beginning of WASM memory in the nativ code, but I don't see a good
reason why not to allow zero address as a parameter value for this function.
2024-03-12 17:46:11 +08:00
Wenyong Huang
0ee5ffce85
Refactor APIs and data structures as preliminary work for Memory64 (#3209)
# Change the data type representing linear memory address from u32 to u64

## APIs signature changes
- (Export)wasm_runtime_module_malloc
  - wasm_module_malloc
    - wasm_module_malloc_internal
  - aot_module_malloc
    - aot_module_malloc_internal
- wasm_runtime_module_realloc
  - wasm_module_realloc
    - wasm_module_realloc_internal
  - aot_module_realloc
    - aot_module_realloc_internal
- (Export)wasm_runtime_module_free
  - wasm_module_free
    - wasm_module_free_internal
  - aot_module_malloc
    - aot_module_free_internal
- (Export)wasm_runtime_module_dup_data
  - wasm_module_dup_data
  - aot_module_dup_data
- (Export)wasm_runtime_validate_app_addr
- (Export)wasm_runtime_validate_app_str_addr
- (Export)wasm_runtime_validate_native_addr
- (Export)wasm_runtime_addr_app_to_native
- (Export)wasm_runtime_addr_native_to_app
- (Export)wasm_runtime_get_app_addr_range
- aot_set_aux_stack
- aot_get_aux_stack
- wasm_set_aux_stack
- wasm_get_aux_stack
- aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert
  and jit_check_app_addr_and_convert
- wasm_exec_env_set_aux_stack
- wasm_exec_env_get_aux_stack
- wasm_cluster_create_thread
- wasm_cluster_allocate_aux_stack
- wasm_cluster_free_aux_stack

## Data structure changes
- WASMModule and AOTModule
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMExecEnv
  - field aux_stack_boundary and aux_stack_bottom
- AOTCompData
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMMemoryInstance(AOTMemoryInstance)
  - field memory_data_size and change __padding to is_memory64
- WASMModuleInstMemConsumption
  - field total_size and memories_size
- WASMDebugExecutionMemory
  - field start_offset and current_pos
- WASMCluster
  - field stack_tops

## Components that are affected by the APIs and data structure changes
- libc-builtin
- libc-emcc
- libc-uvwasi
- libc-wasi
- Python and Go Language Embedding
- Interpreter Debug engine
- Multi-thread: lib-pthread, wasi-threads and thread manager
2024-03-12 11:38:50 +08:00
Wenyong Huang
b6216a5f8a
Fix ip (bytecode offset) not committed into the latest aot frame (#3213) 2024-03-11 18:11:43 +08:00
Enrico Loparco
7692f32a94
Allow overriding max memory on module instantiation (#3198)
This PR adds a max_memory_pages parameter to module instantiation APIs,
to allow overriding the max memory defined in the WASM module.

Sticking to the max memory defined in the module is quite limiting when
using shared memory in production. If targeted devices have different
memory constraints, many wasm files have to be generated with different
max memory values. And device constraints may not be known in advance.

Being able to set the max memory value during module instantiation allows
to reuse the same wasm module, e.g. by retrying instantiation with different
max memory value.
2024-03-05 17:53:26 +08:00
YAMAMOTO Takashi
0e8d949440
lldb_function_to_function_dbi: A hack to avoid crashing on C++ methods (#3190)
Also, print the function name on argument mismatch.
2024-03-04 17:08:11 +08:00
TianlongLiang
cd63b3b8f2
Fix LLVM assertion failure and update CONTRIBUTING.md (#3197)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3188.
2024-03-04 09:01:40 +08:00
Xu Jun
21819fcff6
Fix dynamic offset in BR for block with return type (#3192)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3170.
2024-03-01 10:15:17 +08:00
YAMAMOTO Takashi
503c9694c8
lldb_function_to_function_dbi: Fix a null dereference (#3189)
C++ allows unnamed arguments.
In the debug info, they are represented as DW_TAG_formal_parameter
w/o DW_AT_name.
variable.GetName() here returns NULL for them.

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/3187
2024-02-29 14:03:49 +08:00
Marcin Kolny
bc4f8ab0a5
Specify language in the wasi socket ext project (#3183)
If the language is not specified, CMake will try to find C++ compiler, even
though it is not really needed in that case (as the project is only written in C).
2024-02-28 19:35:00 +08:00
dongsheng28849455
4f6d70bc52
Use indirect call in pre-checker function to avoid relocation in XIP mode (#3142)
The stack profiler `aot_func#xxx` calls the wrapped function of `aot_func_internal#xxx`
by using symbol reference,  but in some platform like xtensa, it’s translated into a native
long call, which needs to resolve the indirect address by relocation and breaks the XIP
feature which requires the eliminating of relocation.

The solution is to change the symbol reference into an indirect call through the lookup
table, the code will be like this:
```llvm
call_wrapped_func:                                ; preds = %stack_bound_check_block
  %func_addr1 = getelementptr inbounds ptr, ptr %func_ptrs_ptr, i32 75
  %func_tmp2 = load ptr, ptr %func_addr1, align 4
  tail call void %func_tmp2(ptr %exec_env)
  ret void
```
2024-02-27 11:17:57 +08:00
Wenyong Huang
2349df1271
Fix ref.func opcode check when GC is enabled (#3181)
The current code assumes that the element type of table segment can
be `funcref` only, but when GC is enabled, the type can be `(ref func)`
also.

Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/3168.
2024-02-27 09:31:19 +08:00
Wenyong Huang
169e164815
Fix wasm loader handling opcode br_table (#3176)
Fix the errors reported in the sanitizer test of nightly run CI.
When the stack is in polymorphic state, the stack operands may be changed
after pop and push operations (e.g. stack is empty but pop op can succeed
in polymorphic, and the push op can push a new operand to stack), this may
impact the following checks to other target blocks of the br_table opcode.
2024-02-23 14:56:20 +08:00
Huang Qi
94db327f06
Add comments to suppress warning from wamrc (#3175) 2024-02-22 18:57:00 +08:00
Huang Qi
1429d8cc03
Fix inconsistent coding convention (#3171) 2024-02-22 10:40:50 +08:00
TianlongLiang
58c980c4df
Fix llvm jit push funcref/externref result type issue (#3169)
When dealing with non-gc enabled funcref/externref, need to make sure
to push them as i32 type.
2024-02-21 12:15:18 +08:00
Jämes Ménétrey
8b8c59589d
Clarify how to verify SGX evidence without an Intel SGX-enabled platform (#3158)
This PR encompasses two complementing purposes:

A documentation on verifying an Intel SGX evidence as produced by WAMR,
including a guide for verification without an Intel SGX-enabled platform.
This also contains a small addition to the RA sample to extract specific
information, such as whether the enclave is running in debug mode.

A C# sample to verify evidence on trusted premises (and without Intel SGX).
Evidence is generated on untrusted environments, using Intel SGX.
2024-02-17 18:44:22 +08:00
Wenyong Huang
3a0e86454e
fast-interp: Fix GC opcode ref.as_non_null (#3156)
The issue was found in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3151.
2024-02-17 11:54:49 +08:00
YAMAMOTO Takashi
b0c54c8a86
Fix a ubsan complaint "applying zero offset to null pointer" (#3160)
Set a type's `result_ref_type_maps` only when `ref_type_map_count > 0`.
2024-02-17 10:11:10 +08:00
Marcin Kolny
e792c35822
Fix null pointer access in fast-interp when configurable soft bound check is enabled (#3150)
The wasm_interp_call_func_bytecode is called for the first time with the empty
module/exec_env to generate a global_handle_table. Before that happens though,
the function checks if the module instance has bounds check enabled. Because
the module instance is null, the program crashes. This PR added an extra check to
prevent the crashes.
2024-02-14 17:18:37 +08:00
Xu Jun
12f834aebd
fast-interp: Fix frame_offset overflow issue (#3149)
The issue was found in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3137.
2024-02-08 11:32:17 +08:00