Commit Graph

317 Commits

Author SHA1 Message Date
Wenyong Huang
f143344842
libc-builtin: Enhance buffered print for printf_wrapper (#3460)
Move print_buf and print_buf to str_context struct to support multi-threading
when BUILTIN_LIBC_BUFFERED_PRINTF macro is enabled, and fix the issue of
losing some text.

ps. https://github.com/bytecodealliance/wasm-micro-runtime/issues/3430.
2024-05-22 17:17:36 +08:00
YAMAMOTO Takashi
403f472ff1
wasi: Tweak the configuration for nuttx and explain why (#3451) 2024-05-21 14:42:30 +08:00
Benbuck Nason
7744e84607
ssp_config.h: Fix ifdef for android random api (#3444)
It looks like the preferred preprocessor check for Android is `__ANDROID__`
instead of `ANDROID`:
https://groups.google.com/g/android-ndk/c/cf9_f1SLXls

Change `(defined(ANDROID) && __ANDROID_API__ < 28)` to
`((defined(ANDROID) || defined(__ANDROID__)) && (__ANDROID_API__ < 28))`.
2024-05-18 20:23:34 +08:00
liang.he
b2eb7d838d
Update spec test to latest commit (#3293)
- Update spec test cases to commit bc76fd79cfe61033d7f4ad4a7e8fc4f996dc5ba8 on Apr. 3
- Update wabt binary to 1.0.34 to support newer spec cases
- Add comparison between table declared elem type and table elem segment value type
- Add a function to decide whether to execute test cases in a running mode
- Keep using interpreter in GC spec because wat2wasm in wabt can't compile if.wast w/o errors
- Re-factoring threads spec test case processing
- Since wabt 1.0.34 release isn't compatible with ubuntu 20.04, compile it from source code
- Disable CI to run aot multi-module temporarily, and will enable it in another PR
2024-05-17 10:40:47 +08:00
YAMAMOTO Takashi
51ecfd6673
libc-wasi: Fix a locking botch (#3437) 2024-05-15 15:41:40 +08:00
Benbuck Nason
c85bada2a9
Add wasm module global type information APIs (#3406)
Support getting global type from `wasm_runtime_get_import_type` and
`wasm_runtime_get_export_type`, and add two APIs:

```C
wasm_valkind_t
wasm_global_type_get_valkind(const wasm_global_type_t global_type);

bool
wasm_global_type_get_mutable(const wasm_global_type_t global_type);
```
2024-05-10 09:15:58 +08:00
Benbuck Nason
ca61184ced
Fix some spelling issues (#3385)
Fix some of the spelling issues found by CSpell.
2024-05-06 07:56:48 +08:00
Wenyong Huang
b11dbcba0a
thread mgr: Free aux stack only when it was allocated (#3282)
When thread manager is enabled, the aux stack of exec_env may be allocated
by wasm_cluster_allocate_aux_stack or disabled by setting aux_stack_bottom
as UINTPTR_MAX directly. For the latter, no need to free it.

And fix an issue when paring `--gc-heap-size=n` argument for iwasm, and
fix a variable shadowed warning in fast-jit.
2024-04-08 09:06:49 +08:00
Wenyong Huang
2013f1f7d7
Fix warnings/issues reported in Windows and by CodeQL/Coverity (#3275)
Fix the warnings and issues reported:
- in Windows platform
- by CodeQL static code analyzing
- by Coverity static code analyzing

And update CodeQL script to build exception handling and memory features.
2024-04-07 11:57:31 +08:00
Wenyong Huang
a23fa9f86c
Implement memory64 for classic interpreter (#3266)
Adding a new cmake flag (cache variable) `WAMR_BUILD_MEMORY64` to enable
the memory64 feature, it can only be enabled on the 64-bit platform/target and
can only use software boundary check. And when it is enabled, it can support both
i32 and i64 linear memory types. The main modifications are:

- wasm loader & mini-loader: loading and bytecode validating process 
- wasm runtime: memory instantiating process
- classic-interpreter: wasm code executing process
- Support memory64 memory in related runtime APIs
- Modify main function type check when it's memory64 wasm file
- Modify `wasm_runtime_invoke_native` and `wasm_runtime_invoke_native_raw` to
  handle registered native function pointer argument when memory64 is enabled
- memory64 classic-interpreter spec test in `test_wamr.sh` and in CI

Currently, it supports memory64 memory wasm file that uses core spec
(including bulk memory proposal) opcodes and threads opcodes.

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3091
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3240
https://github.com/bytecodealliance/wasm-micro-runtime/pull/3260
2024-04-02 15:22:07 +08:00
Xu Jinyang
cef88deedb
Add wasi_ephemeral_nn module support (#3241)
Add `wasi_ephemeral_nn` module support with optional cmake variable,
which was mentioned in #3229.
2024-03-21 21:05:34 +08:00
Wenyong Huang
5e2011ca1d
Fix compilation errors on esp-idf platform (#3224)
The issue was reported in #3208.
2024-03-14 21:31:44 +08:00
TianlongLiang
c3e33a96ea
Remove unused argument in wasm_runtime_lookup_function and refactor WASMModuleInstance (#3218)
Remove the unused parameter `signature` from `wasm_runtime_lookup_function`.

Refactor the layout of WASMModuleInstance structure:
- move common data members `c_api_func_imports` and `cur_exec_env` from
  `WASMModuleInstanceExtraCommon` to `WASMModuleInstance`
- In `WASMModuleInstance`, enlarge `reserved[3]` to `reserved[5]` in case that
  we need to add more fields in the future

ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/2530
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3202
2024-03-13 12:28:45 +08:00
Wenyong Huang
0ee5ffce85
Refactor APIs and data structures as preliminary work for Memory64 (#3209)
# Change the data type representing linear memory address from u32 to u64

## APIs signature changes
- (Export)wasm_runtime_module_malloc
  - wasm_module_malloc
    - wasm_module_malloc_internal
  - aot_module_malloc
    - aot_module_malloc_internal
- wasm_runtime_module_realloc
  - wasm_module_realloc
    - wasm_module_realloc_internal
  - aot_module_realloc
    - aot_module_realloc_internal
- (Export)wasm_runtime_module_free
  - wasm_module_free
    - wasm_module_free_internal
  - aot_module_malloc
    - aot_module_free_internal
- (Export)wasm_runtime_module_dup_data
  - wasm_module_dup_data
  - aot_module_dup_data
- (Export)wasm_runtime_validate_app_addr
- (Export)wasm_runtime_validate_app_str_addr
- (Export)wasm_runtime_validate_native_addr
- (Export)wasm_runtime_addr_app_to_native
- (Export)wasm_runtime_addr_native_to_app
- (Export)wasm_runtime_get_app_addr_range
- aot_set_aux_stack
- aot_get_aux_stack
- wasm_set_aux_stack
- wasm_get_aux_stack
- aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert
  and jit_check_app_addr_and_convert
- wasm_exec_env_set_aux_stack
- wasm_exec_env_get_aux_stack
- wasm_cluster_create_thread
- wasm_cluster_allocate_aux_stack
- wasm_cluster_free_aux_stack

## Data structure changes
- WASMModule and AOTModule
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMExecEnv
  - field aux_stack_boundary and aux_stack_bottom
- AOTCompData
  - field aux_data_end, aux_heap_base and aux_stack_bottom
- WASMMemoryInstance(AOTMemoryInstance)
  - field memory_data_size and change __padding to is_memory64
- WASMModuleInstMemConsumption
  - field total_size and memories_size
- WASMDebugExecutionMemory
  - field start_offset and current_pos
- WASMCluster
  - field stack_tops

## Components that are affected by the APIs and data structure changes
- libc-builtin
- libc-emcc
- libc-uvwasi
- libc-wasi
- Python and Go Language Embedding
- Interpreter Debug engine
- Multi-thread: lib-pthread, wasi-threads and thread manager
2024-03-12 11:38:50 +08:00
Enrico Loparco
7692f32a94
Allow overriding max memory on module instantiation (#3198)
This PR adds a max_memory_pages parameter to module instantiation APIs,
to allow overriding the max memory defined in the WASM module.

Sticking to the max memory defined in the module is quite limiting when
using shared memory in production. If targeted devices have different
memory constraints, many wasm files have to be generated with different
max memory values. And device constraints may not be known in advance.

Being able to set the max memory value during module instantiation allows
to reuse the same wasm module, e.g. by retrying instantiation with different
max memory value.
2024-03-05 17:53:26 +08:00
Marcin Kolny
bc4f8ab0a5
Specify language in the wasi socket ext project (#3183)
If the language is not specified, CMake will try to find C++ compiler, even
though it is not really needed in that case (as the project is only written in C).
2024-02-28 19:35:00 +08:00
Jämes Ménétrey
8b8c59589d
Clarify how to verify SGX evidence without an Intel SGX-enabled platform (#3158)
This PR encompasses two complementing purposes:

A documentation on verifying an Intel SGX evidence as produced by WAMR,
including a guide for verification without an Intel SGX-enabled platform.
This also contains a small addition to the RA sample to extract specific
information, such as whether the enclave is running in debug mode.

A C# sample to verify evidence on trusted premises (and without Intel SGX).
Evidence is generated on untrusted environments, using Intel SGX.
2024-02-17 18:44:22 +08:00
Wenyong Huang
16a4d71b34
Implement GC (Garbage Collection) feature for interpreter, AOT and LLVM-JIT (#3125)
Implement the GC (Garbage Collection) feature for interpreter mode,
AOT mode and LLVM-JIT mode, and support most features of the latest
spec proposal, and also enable the stringref feature.

Use `cmake -DWAMR_BUILD_GC=1/0` to enable/disable the feature,
and `wamrc --enable-gc` to generate the AOT file with GC supported.

And update the AOT file version from 2 to 3 since there are many AOT
ABI breaks, including the changes of AOT file format, the changes of
AOT module/memory instance layouts, the AOT runtime APIs for the
AOT code to invoke and so on.
2024-02-06 20:47:11 +08:00
Enrico Loparco
cfa90ca44f
Use logger for runtime error/debug prints (#3097)
Change runtime internal error/debug prints from using `os_printf()`
to using `LOG_ERROR()`/`LOG_DEBUG()`.
2024-02-06 13:02:54 +08:00
YAMAMOTO Takashi
9afbeab1c8
wasi: Apply wasm_runtime_begin_blocking_op to poll as well (#3080)
While we used a different approach for poll_oneoff [1],
the implementation works only when the poll list includes
an absolute clock event. That is, if we have a thread which is
polling on descriptors without a timeout, we fail to terminate
the thread.

This commit fixes it by applying wasm_runtime_begin_blocking_op
to poll as well.

[1] https://github.com/bytecodealliance/wasm-micro-runtime/pull/1951
2024-01-25 16:29:34 +08:00
Marcin Kolny
1505e61704
Remove a lot of "unused parameter" warnings (#3075)
They might shadow some of the real issues, so better to keep the number
of warnings as low as possible.
2024-01-24 11:21:13 +08:00
YAMAMOTO Takashi
f56154ed80
thread-mgr: Fix locking problems around aux stack allocation (#3073)
Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/3069
2024-01-24 09:46:53 +08:00
Wenyong Huang
9f64340529
Fix some issues reported by CodeQL (#3064)
Refer to https://github.com/bytecodealliance/wasm-micro-runtime/pull/2812
and https://github.com/bytecodealliance/wasm-micro-runtime/security/code-scanning?query=pr%3A2812+is%3Aopen
2024-01-23 21:38:30 +08:00
YAMAMOTO Takashi
ab97d543e0
wasm_cluster_destroy_spawned_exec_env: Avoid "invalid exec env" trap (#3068)
Possible alternatives:

* Make wasm_cluster_destroy_spawned_exec_env take two exec_env.
  One for wasm execution and another to specify the target to destroy.

* Make execute functions to switch exec_env as briefly discussed in
  https://github.com/bytecodealliance/wasm-micro-runtime/pull/2047
2024-01-23 15:20:01 +08:00
Wenyong Huang
ec6d9cb6be
Check arguments before calling bh_hash_map_find (#3055)
Check whether the arguments are NULL before calling bh_hash_map_find,
or lots of "HashMap find elem failed: map or key is NULL" warnings may
be dumped. Reported in #3053.
2024-01-19 19:29:21 +08:00
YAMAMOTO Takashi
8b37048823
Do not inherit WASM_SUSPEND_FLAG_BLOCKING from the parent thread (#3051)
It seems that some users want to wrap rather large chunk of code
with wasm_runtime_begin_blocking_op/wasm_runtime_end_blocking_op.

If the wrapped code happens to have a call to
e.g. wasm_runtime_spawn_exec_env, WASM_SUSPEND_FLAG_BLOCKING is
inherited to the child exec_env and it may cause unexpected behaviors.
2024-01-19 09:55:01 +08:00
tkernelcn
a9807058a6
Set CONFIG_HAS_CAP_ENTER to support posix file api for freertos (#3041)
Since there is no so rich api in freertos like embedded system, simply set
CONFIG_HAS_CAP_ENTER to 1 to support posix file api for freertos.
Test file api in wasm app pass.
2024-01-17 21:20:35 +08:00
liang.he
5c8b8a17a6
Enhancements on wasm function execution time statistic (#2985)
Enhance the statistic of wasm function execution time, or the performance
profiling feature:
- Add os_time_thread_cputime_us() to get the cputime of a thread,
  and use it to calculate the execution time of a wasm function
- Support the statistic of the children execution time of a function,
  and dump it in wasm_runtime_dump_perf_profiling
- Expose two APIs:
  wasm_runtime_sum_wasm_exec_time
  wasm_runtime_get_wasm_func_exec_time

And rename os_time_get_boot_microsecond to os_time_get_boot_us.
2024-01-17 09:51:54 +08:00
Huang Qi
915adc433d
Rename rwlock_init to avoid conflict (#3016)
This symbol is too simple to cause conflict with others, such as the RW
spinlock in Linux kernel and NuttX, so rename it to avoid the conflict.
2024-01-16 08:53:18 +08:00
Enrico Loparco
ff25110840
Return stack frames of crashed thread when using wasm-c-api (#2908)
When using the wasm-c-api and there's a trap, `wasm_func_call()` returns
a `wasm_trap_t *` object. No matter which thread crashes, the trap contains
the stack frames of the main thread.

With this PR, when there's an exception, the stack frames of the thread
where the exception occurs are stored into the thread cluster.
`wasm_func_call()` can then return those stack frames.
2024-01-11 12:13:05 +08:00
Wenyong Huang
c39214e8a5
Fix potential recursive lock in pthread_create_wrapper (#2980)
Potential recursive lock occurs in:
```
pthread_create_wrapper   (acquire exec_env->wait_lock)
  => wasm_cluster_create_thread
    => allocate_aux_stack
      => wasm_runtime_module_malloc_internal
        => wasm_call_function
          => wasm_exec_env_set_thread_info (acquire exec_env->wait_lock again)
```
Allocate aux stack before calling wasm_cluster_create_thread to resolve it.

Reported in https://github.com/bytecodealliance/wasm-micro-runtime/pull/2977.
2024-01-08 09:43:31 +08:00
tkernelcn
4a1ad9a160
freertos: Change ssp_config.h due to clock_nanosleep() not supported in freertos (#2979) 2024-01-04 20:01:54 +08:00
tkernelcn
a2751903ff
freertos: Minor changes for freertos libc_wasi build adaption (#2973) 2024-01-04 09:49:24 +08:00
Yage Hu
bc2d8959dd
Handle ambiguous fstflags on fd_filestat_set_times (#2892)
It's possible to set both `atim` and `atim_now` in the `fstflags`
parameter.  Same goes for `mtin` and `mtim_now`.  However, it's
ambiguous which time should be set in these two cases.  This commit
checks this and returns `EINVAL`.
2023-12-11 14:16:58 +08:00
Wenyong Huang
b008ab4fba
Fix possible dead lock in wasm_cluster_spawn_exec_env (#2882)
Fix a deadlock issue like:

wasm_runtime_spawn_exec_env
wasm_cluster_spawn_exec_env (hold cluster->lock)
wasm_runtime_instantiate_internal
wasm_instantiate
execute_post_instantiate_functions
wasm_exec_env_set_module_inst (grab the lock again)

Refer to #2879.
2023-12-08 17:28:57 +08:00
Yage Hu
0b332d8987
Portably handle fd_advise on directory fd (#2875)
This commit adds a check to `fd_advise`.  If the fd is a directory,
return `ebadf`.  This brings iwasm in line with Wasmtime's behavior.
WASI folks have stated that fd_advise should not work on directories
as this is a Linux-specific behavior:
https://github.com/bytecodealliance/wasmtime/issues/6505#issuecomment-1574122949
2023-12-07 09:52:53 +08:00
Marcin Kolny
5f7079f0f5
Return error when shutdown() fails (#2801)
Fix issue reported in #2787.
2023-11-24 21:03:59 +08:00
Jämes Ménétrey
f9e8b9535e
Attestation: Free JSON from the Wasm module heap (#2803)
The JSON evidence is allocated on the module instance heap, but no API
was given to dispose of this memory buffer. The sample mentions using
the function free, which behaves differently depending on the
execution context.

This fix provides a new function called librats_dispose_evidence_json,
enabling freeing the JSON evidence directly from the Wasm app.
2023-11-22 10:48:14 +08:00
zoraaver
b39fd516d3
Use next generation crypto API on Windows (#2769)
CryptGenRandom is deprecated by Microsoft and may be removed in future
releases. They recommend to use the next generation API instead. See
https://learn.microsoft.com/en-us/windows/win32/seccng/cng-portal for
more details. Also, refactor the random functions to return error codes
rather than aborting the program if they fail.
2023-11-17 18:40:29 +08:00
zoraaver
8c27676718
Return uint32 from WASI functions (#2749)
Returning uint16 from WASI functions is technically correct. However,
the smallest integer type in WASM is int32 and since we don't guarantee
that the upper 16 bits of the result are zero'ed, it can result in
tricky bugs if the language SDK being used in the WASM app does not cast
back immediately to uint16. To prevent this, we directly return uint32
instead, so that the result is well-defined as a 32-bit number.
2023-11-13 20:07:32 +08:00
Haiyue Wang
625c95816d
Fix switch-case fallthrough compilation warning (#2753)
The commit fa5e9d72b0 ("Abstract POSIX filesystem functions") introduces
the build warning:

./core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c: In function ‘fd_object_release’:
./core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:545:20: warning: this statement may fall through [-Wimplicit-fallthrough=]
  545 |                 if (os_is_dir_stream_valid(&fo->directory.handle)) {
      |                    ^
./core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:549:13: note: here
  549 |             default:
      |             ^~~~~~~

Refer to the commit fb4afc7ca4 ("Apply clang-format for core/iwasm compilation and libraries"),
add one line "// Fallthrough." to make compiler happy.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
2023-11-13 15:01:13 +08:00
Wenyong Huang
71340b79a6
Merge pull request #2740 from bytecodealliance/dev/wasi-libc-windows
The implementation is already in a stage where it's possible to compile WAMR
with wasi libc enabled and run wasi modules without errors.
2023-11-10 16:58:31 +08:00
YAMAMOTO Takashi
24c4d256b3
Grab cluster->lock when modifying exec_env->module_inst (#2685)
Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/2680

And when switching back to the original module_inst, propagate exception if any.

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/2512
2023-11-09 18:56:02 +08:00
Wenyong Huang
20f1a8c86c Merge branch main into dev/wasi-libc-windows 2023-11-09 10:13:59 +08:00
zoraaver
3c9cd40aa6
Provide default vprintf on UWP (#2725)
UWP apps do not have a console attached so any output to stdout/stderr
is lost. Therefore, provide a default BH_VPRINTF in that case for debug
builds which redirects output to the debugger.
2023-11-07 20:37:43 +08:00
zoraaver
13875f43c6
Enable WASI tests on Windows CI (#2699)
Most of the WASI filesystem tests require at least creating/deleting a
file to test filesystem functionality so some additional filesystem APIs
have been implemented on Windows so we can test what has been
implemented so far. For those WASI functions which haven't been
implemented, we skip the tests. These will be implemented in a future PR
after which we can remove the relevant filters.

Additionally, in order to run the WASI socket and thread tests, we need
to install the wasi-sdk in CI and build the test source code prior to
running the tests.
2023-11-06 19:24:06 +08:00
Gavin Hayes
51a6b069d2
Fix use getrandom on cosmopolitan libc (#2674)
Fixes the Cosmopolitan Libc platform attempting to use `/dev/urandom`
on operating systems that do not have it.

Signed-off-by: G4Vi <gavin@dylibso.com>
2023-10-27 14:07:16 +08:00
zoraaver
e7a62d2099
Refactor clock functions to use WASI types (#2666)
Refactoring the clock functions to use WASI types so we can simplify the
code and remove some unnecessary boilerplate. See
https://github.com/bytecodealliance/wasm-micro-runtime/pull/2637#discussion_r1362202879
for details.
2023-10-25 18:06:04 +08:00
Wenyong Huang
9d7931e1d2
Fix several typo/warning/unused-code issues (#2655)
- Fix typo in wamr-test-suites script
- Fix compilation warnings in libc-wasi posix.c
- Remove unused code fast-jit jit_frontend.c
- Remove duplicated exception print in `iwasm -f <function>`
- Fix return value in void function wasm_runtime_set_wasi_ctx
2023-10-24 09:19:45 +08:00
fadumina1
a874bf0ff8
Implement wasi clock_time/clock_res get (#2637)
Add os_clock_res_get and os_clock_time_get in platform_api_extension.h,
and implement them in posix like platforms and windows platform.
2023-10-22 21:11:29 +08:00