Commit Graph

2030 Commits

Author SHA1 Message Date
Wenyong Huang
f933d4c829
loader: Remove updating ctx->dynamic_offset in emit_br_info (#3259)
Should not update `ctx->dynamic_offset` in emit_br_info, since the `Part e` only
sets the dst offsets, the operand stack should not be changed, e.g., the stack
operands are to be used by the opcodes followed by `br_if` opcode.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3210.
2024-03-28 16:07:02 +08:00
Wenyong Huang
89f4982123
Fix compilation errors on zephyr platform (#3255)
Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/3252.
2024-03-26 17:55:53 +08:00
liang.he
498eb5d54a
Append \0 to every name string in aot name section (#3249)
Since strings in .name section in .wasm is not c-style, need to append
a `\0` to each string in .name section in AOT file when emitting.
2024-03-26 14:27:20 +08:00
liang.he
d8d8f8ce04
Implement apis to set and get the name of a wasm module (#3254)
Add API wasm_runtime_set_module_name and wasm_runtime_get_module_name,
and by default, a module's name is "" if the set module name api isn't called.
2024-03-26 12:10:13 +08:00
Wenyong Huang
ca364eb5d7
Fix nightly-run CI and CodeQL CI (#3250)
- Fix nightly run CI failure which was introduced by PR #3239 and now it must set
  WAMR_BUILD_TARGET when building iwasm for Android platform
- Remove building llvm, wamrc and jit in CodeQL CI, since it will
  do static code analyzing for llvm project and cause CodeQL run
  failed: `Oops! A fatal internal error occurred.
  This particular kind of error most often happens as a side effect of running out of
  disk space.`
2024-03-22 14:29:30 +08:00
Wenyong Huang
6a55bde5b3
Add issue templates (#3248)
Add issue templates of blank issue, improvement and reporting bug in WAMR.
And fix several invalid links in ATTRIBUTIONS.md.
2024-03-22 13:49:39 +08:00
TianlongLiang
64b6c688a2
posix_file.c: Correct the dirfd argument that passes to fstatat (#3244)
This PR fixes the random failing test case `nofollow_errors` mentioned in
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3222

```C
// dirfd: This is the file descriptor of the directory relative to which the pathname is interpreted. 
int openat(int dirfd, const char *pathname, int flags, ...);
```
The value should be a directory handle instead of a file handle (which is always -1 in this context)
returned from `openat`.
2024-03-22 10:55:39 +08:00
Yo Han Joo
b4941b0cde
Go binding: Change C.long to C.int64_t when call wasm_runtime_set_wasi_args_ex (#3235)
- Change `C.long` to `C.int64_t` due to error:
```sh
./module.go:119:64: cannot use _Ctype_long(stdinfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
./module.go:120:43: cannot use _Ctype_long(stdoutfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
./module.go:120:60: cannot use _Ctype_long(stderrfd) (value of type _Ctype_long) as _Ctype_longlong value in variable declaration
```
- Change offset from `uint32` to `uint64` due to casting error

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3220
https://stackoverflow.com/questions/70243683/how-to-convert-c-uint64-t-to-cgo-consistently-across-os
2024-03-22 10:45:14 +08:00
Xu Jinyang
cef88deedb
Add wasi_ephemeral_nn module support (#3241)
Add `wasi_ephemeral_nn` module support with optional cmake variable,
which was mentioned in #3229.
2024-03-21 21:05:34 +08:00
Wenyong Huang
e003ee1e29
CodeQL: Add more build combinations and disable run on PR (#3246)
Enhance CodeQL Code Security Analysis:
- Add more compilation combinations to build iwasm with different kinds of features
- Disable run on PR created and keep nightly run, since the whole time is very long,
   and will check how to restore run on PR created in the future
2024-03-21 14:18:27 +08:00
Brian
a86eeb273c
Add CodeQL Workflow for Code Security Analysis (#2812)
Add CodeQL Workflow for Code Security Analysis

This pull request introduces a CodeQL workflow to enhance the security analysis of our repository.
CodeQL is a powerful static analysis tool that helps identify and mitigate security vulnerabilities in
our codebase. By integrating this workflow into our GitHub Actions, we can proactively identify
and address potential issues before they become security threats.

We added a new CodeQL workflow file (.github/workflows/codeql.yml) that
- Runs on nightly-run, and consider runs on every pull request to the main branch in the future.
- Excludes queries with a high false positive rate or low-severity findings.
- Does not display results for third-party code, focusing only on our own codebase.

Testing:
To validate the functionality of this workflow, we have run several test scans on the codebase and
reviewed the results. The workflow successfully compiles the project, identifies issues, and provides
actionable insights while reducing noise by excluding certain queries and third-party code.

Deployment:
Once this pull request is merged, the CodeQL workflow will be active and automatically run on
every push and pull request to the main branch. To view the results of these code scans, please
follow these steps:
1. Under the repository name, click on the Security tab.
2. In the left sidebar, click Code scanning alerts.

Additional Information:
- You can further customize the workflow to adapt to your specific needs by modifying the workflow file.
- For more information on CodeQL and how to interpret its results, refer to the GitHub documentation
and the CodeQL documentation.

Signed-off-by: Brian <bayuan@purdue.edu>
2024-03-21 12:37:47 +08:00
Wenyong Huang
76254183f9
Make android platform's cmake flags configurable (#3239)
Don't hardcode the cmake configurations in the Android platform's CMakeLists.txt.

Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/3238
2024-03-21 11:08:40 +08:00
Tao Xiong
29d83224a8
Add esp32c6 support (#3234)
This PR adds support for ESP32 C6, which has been mentioned in #3208.
2024-03-19 08:15:46 +08:00
Liangyu Zhang
b11a1d157d
GC: Add wasm_struct_obj_get_field_count API (#3236) 2024-03-18 16:26:30 +08:00
TianlongLiang
7486056aee
Fix nightly run tsan ASLR issue (#3233)
The nightly run CI reported error:
  "FATAL: ThreadSanitizer: unexpected memory mapping 0x5be565bf3000-0x5be565bfb000"
which is caused by the ASLR issue, we set `vm.mmap_rnd_bits` to 28 to resolve it,
according to the post below:
https://stackoverflow.com/questions/77850769/fatal-threadsanitizer-unexpected-memory-mapping-when-running-on-linux-kernels

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/8319242277/job/22762363873#step:14:2008
2024-03-18 11:15:47 +08:00
liang.he
8c1269d44d
trans_wasm_func_name.py: Correct function index during translation (#3232)
Adding the N from "aot_func#N" with the import function count is the correct
wasm function index.
2024-03-18 10:32:55 +08:00
Wenyong Huang
ff296c1a62
Fix aot relocation symbols not found on windows 32-bit (#3231)
The symbols in windows 32-bit may start with '_' and can not be found
when resolving the relocations to them. This PR ignores the underscore
when handling the relocation name of AOT_FUNC_INTERNAL_PREFIX, and
redirect the relocation with name "_aot_stack_sizes" to the relocation with
name ".aot_stack_sizes" (the name of the data section created).

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3216
2024-03-18 09:51:38 +08:00
Wenyong Huang
5e2011ca1d
Fix compilation errors on esp-idf platform (#3224)
The issue was reported in #3208.
2024-03-14 21:31:44 +08:00
TianlongLiang
de803b2beb
Small refactor on WASMModuleInstance and fix Go/Python language bindings (#3227)
- Merge unused field `used_to_be_wasi_ctx` in `AOTModuleInstance` into `reserved` area
- Add field `memory_lock` in `WASMMemoryInstance` for future refactor
- Go binding: fix type error
    https://github.com/bytecodealliance/wasm-micro-runtime/issues/3220
- Python binding:
    type annotation uses the union operator "|", which requires Python version >=3.10
2024-03-14 15:30:28 +08:00
TianlongLiang
c3e33a96ea
Remove unused argument in wasm_runtime_lookup_function and refactor WASMModuleInstance (#3218)
Remove the unused parameter `signature` from `wasm_runtime_lookup_function`.

Refactor the layout of WASMModuleInstance structure:
- move common data members `c_api_func_imports` and `cur_exec_env` from
  `WASMModuleInstanceExtraCommon` to `WASMModuleInstance`
- In `WASMModuleInstance`, enlarge `reserved[3]` to `reserved[5]` in case that
  we need to add more fields in the future

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/2530
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3202
2024-03-13 12:28:45 +08:00
Marcin Kolny
ce44e0ec0c
Allow converting the zero wasm address to native (#3215)
This allows to know the beginning of the wasm address space. At the moment
to achieve that, we need to apply a `hack wasm_runtime_addr_app_to_native(X)-X`
to get the beginning of WASM memory in the nativ code, but I don't see a good
reason why not to allow zero address as a parameter value for this function.
2024-03-12 17:46:11 +08:00
lucianoiam
df57e70437
Fix compilation errors on MinGW (#3217)
Before PR compilation failed because of two errors:

1 - Usage of CMake target_link_libraries()

Make Error at CMakeLists.txt:154 (target_link_libraries):
  The keyword signature for target_link_libraries has already been used with
  the target "iwasm_shared".  All uses of target_link_libraries with a target
  must be either all-keyword or all-plain.

  The uses of the keyword signature are here:

   * CMakeLists.txt:148 (target_link_libraries)

See https://stackoverflow.com/questions/47737558/uses-of-target-link-libraries-must-be-either-all-keyword-or-all-plain

Fixed by adding keyword INTERFACE

2 - Undefined symbols during linkage, fixed by adding -lwsock32.
2024-03-12 17:38:07 +08:00
Wenyong Huang
0ee5ffce85
Refactor APIs and data structures as preliminary work for Memory64 (#3209)
# Change the data type representing linear memory address from u32 to u64

## APIs signature changes
- (Export)wasm_runtime_module_malloc
  - wasm_module_malloc
    - wasm_module_malloc_internal
  - aot_module_malloc
    - aot_module_malloc_internal
- wasm_runtime_module_realloc
  - wasm_module_realloc
    - wasm_module_realloc_internal
  - aot_module_realloc
    - aot_module_realloc_internal
- (Export)wasm_runtime_module_free
  - wasm_module_free
    - wasm_module_free_internal
  - aot_module_malloc
    - aot_module_free_internal
- (Export)wasm_runtime_module_dup_data
  - wasm_module_dup_data
  - aot_module_dup_data
- (Export)wasm_runtime_validate_app_addr
- (Export)wasm_runtime_validate_app_str_addr
- (Export)wasm_runtime_validate_native_addr
- (Export)wasm_runtime_addr_app_to_native
- (Export)wasm_runtime_addr_native_to_app
- (Export)wasm_runtime_get_app_addr_range
- aot_set_aux_stack
- aot_get_aux_stack
- wasm_set_aux_stack
- wasm_get_aux_stack
- aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert
  and jit_check_app_addr_and_convert
- wasm_exec_env_set_aux_stack
- wasm_exec_env_get_aux_stack
- wasm_cluster_create_thread
- wasm_cluster_allocate_aux_stack
- wasm_cluster_free_aux_stack

## Data structure changes
- WASMModule and AOTModule
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMExecEnv
  - field aux_stack_boundary and aux_stack_bottom
- AOTCompData
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMMemoryInstance(AOTMemoryInstance)
  - field memory_data_size and change __padding to is_memory64
- WASMModuleInstMemConsumption
  - field total_size and memories_size
- WASMDebugExecutionMemory
  - field start_offset and current_pos
- WASMCluster
  - field stack_tops

## Components that are affected by the APIs and data structure changes
- libc-builtin
- libc-emcc
- libc-uvwasi
- libc-wasi
- Python and Go Language Embedding
- Interpreter Debug engine
- Multi-thread: lib-pthread, wasi-threads and thread manager
2024-03-12 11:38:50 +08:00
Wenyong Huang
b6216a5f8a
Fix ip (bytecode offset) not committed into the latest aot frame (#3213) 2024-03-11 18:11:43 +08:00
Enrico Loparco
f550feb039
Demangle function names in stack trace when using addr2line script (#3211)
Last bit missing from #3206: demangling of function names (useful for wasm
generated from Rust for instance) using `llvm-cxxfilt`.

Before this PR:
```text
0: abort
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/panic_abort/src/lib.rs:85
1: _ZN3std3sys4wasi14abort_internal17h50698daab05bf73bE
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/wasi/mod.rs:181
2: _ZN3std7process5abort17h6bc522b6749f17cfE
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/process.rs:2278
3: _ZN3std5alloc8rust_oom17h452ad5ba6cebff96E
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:364
```

After:
```text
0: abort
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/panic_abort/src/lib.rs:85
1: std::sys::wasi::abort_internal::h50698daab05bf73b
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/wasi/mod.rs:181
2: std::process::abort::h6bc522b6749f17cf
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/process.rs:2278
3: std::alloc::rust_oom::h452ad5ba6cebff96
        at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:364
```
2024-03-11 14:27:09 +08:00
Enrico Loparco
0e4c4799b1
Get location info from function indexes in addr2line script (#3206)
Update the `addr2line` script so that:
- line info is printed in a more convenient format, e.g.
```
0: c
        at wasm-micro-runtime/test-tools/addr2line/trap.c:5:1
1: b
        at wasm-micro-runtime/test-tools/addr2line/trap.c:11:12
2: a
        at wasm-micro-runtime/test-tools/addr2line/trap.c:17:12
```
similar to how Rust prints stack traces when there's a panic. In an IDE, the user
can conveniently click on the printed path and be redirected to the file line.
- a new `--no-addr` argument can be provided to the script 

It can be used in fast interpreter mode (that is not supported by the script otherwise)
or with older wamr versions (where the stack trace only had the function index info
and not the function address). In that case, `wasm-objdump` is used to get the function
name from the index and `llvm-dwarfdump` to obtain the location info (where the line
refers to the start of the function).
2024-03-08 10:20:04 +08:00
Wenyong Huang
a43018ff72
Update document to add wamr-rust-sdk introduction (#3204)
Merge branch dev/rust_sdk to main.
The wamr-rust-sdk has been migrated to the standalone repo:
  https://github.com/bytecodealliance/wamr-rust-sdk
So here we just update the document to add the introduction.
2024-03-06 09:29:40 +08:00
liang.he
d555c16d11
Revert PR #3194 (#3199)
- Address values in call stack dump are relative to file beginning
- If running under fast-interp mode, address values are relative to
  every pre-compiled function beginning, which is not compatible
  with addr2line
2024-03-05 18:13:33 +08:00
Enrico Loparco
7692f32a94
Allow overriding max memory on module instantiation (#3198)
This PR adds a max_memory_pages parameter to module instantiation APIs,
to allow overriding the max memory defined in the WASM module.

Sticking to the max memory defined in the module is quite limiting when
using shared memory in production. If targeted devices have different
memory constraints, many wasm files have to be generated with different
max memory values. And device constraints may not be known in advance.

Being able to set the max memory value during module instantiation allows
to reuse the same wasm module, e.g. by retrying instantiation with different
max memory value.
2024-03-05 17:53:26 +08:00
YAMAMOTO Takashi
0e8d949440
lldb_function_to_function_dbi: A hack to avoid crashing on C++ methods (#3190)
Also, print the function name on argument mismatch.
2024-03-04 17:08:11 +08:00
Wenyong Huang
01575fc6da
Download jetstream src from github instead of browserbench.org (#3196)
Downloading benchmark jetstream's source files one by one from
https://browserbench.org by using wget is time consuming and not
stable, this PR updates the scripts to download the files from
https://github.com/mozilla/perf-automation.
2024-03-04 14:20:07 +08:00
TianlongLiang
cd63b3b8f2
Fix LLVM assertion failure and update CONTRIBUTING.md (#3197)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3188.
2024-03-04 09:01:40 +08:00
liang.he
1b9fbb162f
addr2line.py: Fix issue of applying offset in call stacks information (#3194)
The offset value is based on the start of the wasm file, it also equals to
the value of `wasm-objdump -d <wasm_file>`.
2024-03-01 10:56:36 +08:00
Xu Jun
21819fcff6
Fix dynamic offset in BR for block with return type (#3192)
The issue was reported in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3170.
2024-03-01 10:15:17 +08:00
Wenyong Huang
5635244169
Remove unused core/app-mgr folder (#3186)
The app-manager and app-framework have been migrated to:
https://github.com/bytecodealliance/wamr-app-framework
2024-02-29 16:16:02 +08:00
YAMAMOTO Takashi
503c9694c8
lldb_function_to_function_dbi: Fix a null dereference (#3189)
C++ allows unnamed arguments.
In the debug info, they are represented as DW_TAG_formal_parameter
w/o DW_AT_name.
variable.GetName() here returns NULL for them.

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/3187
2024-02-29 14:03:49 +08:00
Marcin Kolny
bc4f8ab0a5
Specify language in the wasi socket ext project (#3183)
If the language is not specified, CMake will try to find C++ compiler, even
though it is not really needed in that case (as the project is only written in C).
2024-02-28 19:35:00 +08:00
mkolchurin
c949f3d2aa
zephyr: Implement Alloc_With_System_Allocator (#3179)
Add zephyr libc malloc/realloc/free which were introduced since version 1.13.0.
2024-02-28 11:18:23 +08:00
zoraaver
92bd3ba17d
Implement the remaining Windows filesystem functions (#3166)
Now that the filesystem implementation is now complete, the previous
test filters on Windows can be removed. Some of the tests only pass when
certain environment variables have been set on Windows so an extra step
has been added in the wasi test runner script to modify the test config
files before the tests begin.
2024-02-28 11:02:42 +08:00
dongsheng28849455
4f6d70bc52
Use indirect call in pre-checker function to avoid relocation in XIP mode (#3142)
The stack profiler `aot_func#xxx` calls the wrapped function of `aot_func_internal#xxx`
by using symbol reference,  but in some platform like xtensa, it’s translated into a native
long call, which needs to resolve the indirect address by relocation and breaks the XIP
feature which requires the eliminating of relocation.

The solution is to change the symbol reference into an indirect call through the lookup
table, the code will be like this:
```llvm
call_wrapped_func:                                ; preds = %stack_bound_check_block
  %func_addr1 = getelementptr inbounds ptr, ptr %func_ptrs_ptr, i32 75
  %func_tmp2 = load ptr, ptr %func_addr1, align 4
  tail call void %func_tmp2(ptr %exec_env)
  ret void
```
2024-02-27 11:17:57 +08:00
Wenyong Huang
2349df1271
Fix ref.func opcode check when GC is enabled (#3181)
The current code assumes that the element type of table segment can
be `funcref` only, but when GC is enabled, the type can be `(ref func)`
also.

Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/3168.
2024-02-27 09:31:19 +08:00
Wenyong Huang
169e164815
Fix wasm loader handling opcode br_table (#3176)
Fix the errors reported in the sanitizer test of nightly run CI.
When the stack is in polymorphic state, the stack operands may be changed
after pop and push operations (e.g. stack is empty but pop op can succeed
in polymorphic, and the push op can push a new operand to stack), this may
impact the following checks to other target blocks of the br_table opcode.
2024-02-23 14:56:20 +08:00
mkolchurin
88bfbcf89e
zephyr: include math only with minimal libc (#3177)
Use math functions only with `CONFIG_MINIMAL_LIBC=y`.

`CONFIG_PICOLIBC=y` or `CONFIG_NEWLIB_LIBC=y` provides math functions
that are used by wasm, and compilation fails when they are selected.

Signed-off-by: Maxim Kolchurin <maxim.kolchurin@gmail.com>
2024-02-23 11:24:51 +08:00
Huang Qi
94db327f06
Add comments to suppress warning from wamrc (#3175) 2024-02-22 18:57:00 +08:00
Enrico Loparco
8493ffa1cc
Add vprintf override for android and esp-idf (#3174)
And update document.
2024-02-22 17:02:46 +08:00
TianlongLiang
0fa0beba94
VSCode IDE enhancement and readme update (#3172)
- Temporarily disable the deque test for the VS Code extension, as the
  Rust formatter seems to malfunction after a recent VS Code update.
- Add configuration for iwasm host managed heap size, allowing users
  to flexibly `malloc` memory. This also fixes the current bug that when
  default size is 0, it can't run and debug.
- Apply coding style formatting for WAMR IDE source code and add a
  format check for it in CI.
- Update document and some screenshots.
2024-02-22 12:36:49 +08:00
Huang Qi
1429d8cc03
Fix inconsistent coding convention (#3171) 2024-02-22 10:40:50 +08:00
TianlongLiang
58c980c4df
Fix llvm jit push funcref/externref result type issue (#3169)
When dealing with non-gc enabled funcref/externref, need to make sure
to push them as i32 type.
2024-02-21 12:15:18 +08:00
Wenyong Huang
63cd567b3f
Separate app-manager and app-framework from WAMR (#3129)
As planned, the app-manager and app-framework are to be migrated to
https://github.com/bytecodealliance/wamr-app-framework.

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/2329
https://github.com/bytecodealliance/wasm-micro-runtime/wiki/TSC-meeting-notes
2024-02-20 18:12:36 +08:00
mkolchurin
b9db23b983
zephyr: Use zephyr sys_cache instead of CMSIS (#3162)
When running AOT code in Zephyr on STM32H743VIT6 without
CONFIG_CACHE_MANAGEMENT=y, a hard fault occurs, which leads to
SCB_CleanDCache().

It’s better to use the functions built into Zephyr.
2024-02-20 09:34:25 +08:00