This fixes armhf build:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/2315
Note: This is not the only place which seems to have the same problem.
For example, many of examples sharing the similar structure probably
have the same problem. This commit leaves them for now.
Major changes:
- Public headers inside `wasi-nn/include`
- Put cmake files in `cmake` folder
- Make linux iwasm link with `${WASI_NN_LIBS}` so iwasm can enable wasi-nn
LLVM PGO (Profile-Guided Optimization) allows the compiler to better optimize code
for how it actually runs. This PR implements the AOT static PGO, and is tested on
Linux x86-64 and x86-32. The basic steps are:
1. Use `wamrc --enable-llvm-pgo -o <aot_file_of_pgo> <wasm_file>`
to generate an instrumented aot file.
2. Compile iwasm with `cmake -DWAMR_BUILD_STATIC_PGO=1` and run
`iwasm --gen-prof-file=<raw_profile_file> <aot_file_of_pgo>`
to generate the raw profile file.
3. Run `llvm-profdata merge -output=<profile_file> <raw_profile_file>`
to merge the raw profile file into the profile file.
4. Run `wamrc --use-prof-file=<profile_file> -o <aot_file> <wasm_file>`
to generate the optimized aot file.
5. Run the optimized aot_file: `iwasm <aot_file>`.
The test scripts are also added for each benchmark, run `test_pgo.sh` under
each benchmark's folder to test the AOT static pgo.
Segue is an optimization technology which uses x86 segment register to store
the WebAssembly linear memory base address, so as to remove most of the cost
of SFI (Software-based Fault Isolation) base addition and free up a general
purpose register, by this way it may:
- Improve the performance of JIT/AOT
- Reduce the footprint of JIT/AOT, the JIT/AOT code generated is smaller
- Reduce the compilation time of JIT/AOT
This PR uses the x86-64 GS segment register to apply the optimization, currently
it supports linux and linux-sgx platforms on x86-64 target. By default it is disabled,
developer can use the option below to enable it for wamrc and iwasm(with LLVM
JIT enabled):
```bash
wamrc --enable-segue=[<flags>] -o output_file wasm_file
iwasm --enable-segue=[<flags>] wasm_file [args...]
```
`flags` can be:
i32.load, i64.load, f32.load, f64.load, v128.load,
i32.store, i64.store, f32.store, f64.store, v128.store
Use comma to separate them, e.g. `--enable-segue=i32.load,i64.store`,
and `--enable-segue` means all flags are added.
Acknowledgement:
Many thanks to Intel Labs, UC San Diego and UT Austin teams for introducing this
technology and the great support and guidance!
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Co-authored-by: Vahldiek-oberwagner, Anjo Lucas <anjo.lucas.vahldiek-oberwagner@intel.com>
Add nightly (UTC time) checks with asan and ubsan, and also put gcc-4.8 build
to nightly run since we don't need to run it with every PR.
Co-authored-by: Maksim Litskevich <makslit@amazon.co.uk>
Currently, if a thread is spawned and raises an exception after the main thread
has finished, iwasm returns with success instead of returning 1 (i.e. error).
Since wasm_runtime_get_wasi_exit_code waits for all threads to finish and only
returns the wasi exit code, this PR performs the exception check again and
returns error if an exception was raised.
Removes the `west espressif install` command which is no longer
supported.
https://github.com/zephyrproject-rtos/hal_espressif/pull/130
Also adds a reference to the Zephyr SDK, which will install the
necessary toolchains for all supported targets.
Signed-off-by: Daniel Mangum <georgedanielmangum@gmail.com>
In #1928 we added support for GCC 4.8 but we don't continuously test if it's
working. This PR added a GitHub actions job to test compilation on GCC 4.8
for interpreters and Fast JIT (LLVM JIT/AOT might be added in the future).
The compilation is done using ubuntu 14.04 image as that's the simplest way
to get GCC 4.8 compiler. The job only compiles the code but does not run any
tests.
Support collecting code coverage with wamr-test-suites script by using
lcov and genhtml tools, eg.:
cd tests/wamr-test-suites
./test_wamr.sh -s spec -b -P -C
The default code coverage and html files are generated at:
tests/wamr-test-suites/workspace/wamr.lcov
tests/wamr-test-suites/workspace/wamr-lcov.zip
And update wamr-test-suites scripts to support testing GC spec cases to
avoid frequent synchronization conflicts between branch main and dev/gc.
Enable setting running mode when executing a wasm bytecode file
- Four running modes are supported: interpreter, fast-jit, llvm-jit and multi-tier-jit
- Add APIs to set/get the default running mode of the runtime
- Add APIs to set/get the running mode of a wasm module instance
- Add running mode options for iwasm command line tool
And add size/opt level options for LLVM JIT
Current SGX lib-rats wasm module hash is stored in a global buffer,
which may be overwritten if there are multiple wasm module loadings.
We move the module hash into the enclave module to resolve the issue.
And rename the SGX_IPFS macro/variable in Makefile and Enclave.edl to
make the code more consistent.
And refine the sgx-ra sample document.
Enlarge the default wasm operand stack size to 64KB since the original default
size 16KB is a little small, and the operand stack overflow exception is often
thrown when running wasm apps.
The current implementation of remote attestation does not take into
account the integrity of the wasm module. The SHA256 of the wasm
module has been put into user_data to generate the quote, and more
parameters are exposed for further verification.
Refine the generated LLVM IRs at the beginning of each LLVM AOT/JIT function
to fasten the LLVM IR optimization:
- Only create argv_buf if there are func calls in this function
- Only create native stack bound if stack bound check is enabled
- Only create aux stack info if there is opcode set_global_aux_stack
- Only create native symbol if indirect_mode is enabled
- Only create memory info if there are memory operations
- Only create func_type_indexes if there is opcode call_indirect
Add a new options to control the native stack hw bound check feature:
- Besides the original option `cmake -DWAMR_DISABLE_HW_BOUND_CHECK=1/0`,
add a new option `cmake -DWAMR_DISABLE_STACK_HW_BOUND_CHECK=1/0`
- When the linear memory hw bound check is disabled, the stack hw bound check
will be disabled automatically, no matter what the input option is
- When the linear memory hw bound check is enabled, the stack hw bound check
is enabled/disabled according to the value of input option
- Besides the original option `--bounds-checks=1/0`, add a new option
`--stack-bounds-checks=1/0` for wamrc
Refer to: https://github.com/bytecodealliance/wasm-micro-runtime/issues/1677
Add macro WASM_ENABLE_WORD_ALING_READ to enable reading
1/2/4 and n bytes data from vram buffer, which requires 4-byte addr
alignment reading.
Eliminate XIP AOT relocations related to the below ones:
i32_div_u, f32_min, f32_max, f32_ceil, f32_floor, f32_trunc, f32_rint
Allow to unregister (or unload) the previously registered native libs,
so that no need to restart the whole engine by using
`wasm_runtime_destroy/wasm_runtime_init`.
Use the cmake variable `WAMR_BUILD_GLOBAL_HEAP_POOL` and
`WAMR_BUILD_GLOBAL_HEAP_SIZE` to enable/disable the global heap pool
and set its size. And set the default global heap size in core/config.h and
the cmake files.
As a result, the developers who build iwasm can easily enable/disable the
global heap pool and change its size regardless of the iwasm implementation,
without manually finding and patching the right location for that value.
Refactor the layout of interpreter and AOT module instance:
- Unify the interp/AOT module instance, use the same WASMModuleInstance/
WASMMemoryInstance/WASMTableInstance data structures for both interpreter
and AOT
- Make the offset of most fields the same in module instance for both interpreter
and AOT, append memory instance structure, global data and table instances to
the end of module instance for interpreter mode (like AOT mode)
- For extra fields in WASM module instance, use WASMModuleInstanceExtra to
create a field `e` for interpreter
- Change the LLVM JIT module instance creating process, LLVM JIT uses the WASM
module and module instance same as interpreter/Fast-JIT mode. So that Fast JIT
and LLVM JIT can access the same data structures, and make it possible to
implement the Multi-tier JIT (tier-up from Fast JIT to LLVM JIT) in the future
- Unify some APIs: merge some APIs for module instance and memory instance's
related operations (only implement one copy)
Note that the AOT ABI is same, the AOT file format, AOT relocation types, how AOT
code accesses the AOT module instance and so on are kept unchanged.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1384
This PR integrates an Intel SGX feature called Intel Protection File System Library (IPFS)
into the runtime to create, operate and delete files inside the enclave, while guaranteeing
the confidentiality and integrity of the data persisted. IPFS can be referred to here:
https://www.intel.com/content/www/us/en/developer/articles/technical/overview-of-intel-protected-file-system-library-using-software-guard-extensions.html
Introduce a cmake variable `WAMR_BUILD_SGX_IPFS`, when enabled, the files interaction
API of WASI will leverage IPFS, instead of the regular POSIX OCALLs. The implementation
has been written with light changes to sgx platform layer, so all the security aspects
WAMR relies on are conserved.
In addition to this integration, the following changes have been made:
- The CI workflow has been adapted to test the compilation of the runtime and sample
with the flag `WAMR_BUILD_SGX_IPFS` set to true
- Introduction of a new sample that demonstrates the interaction of the files (called `file`),
- Documentation of this new feature
Implement more socket APIs, refer to #1336 and below PRs:
- Implement wasi_addr_resolve function (#1319)
- Fix socket-api byte order issue when host/network order are the same (#1327)
- Enhance sock_addr_local syscall (#1320)
- Implement sock_addr_remote syscall (#1360)
- Add support for IPv6 in WAMR (#1411)
- Implement ns lookup allowlist (#1420)
- Implement sock_send_to and sock_recv_from system calls (#1457)
- Added http downloader and multicast socket options (#1467)
- Fix `bind()` calls to receive the correct size of `sockaddr` structure (#1490)
- Assert on correct parameters (#1505)
- Copy only received bytes from socket recv buffer into the app buffer (#1497)
Co-authored-by: Marcin Kolny <mkolny@amazon.com>
Co-authored-by: Marcin Kolny <marcin.kolny@gmail.com>
Co-authored-by: Callum Macmillan <callumimacmillan@gmail.com>
Use the semantic versioning (https://semver.org) to replace the current date
versioning system, which is more general and is requested by some developers,
e.g. issue #1357.
There are three parts in the new version string:
- major. Any incompatible modification on ABIs and APIs will lead to an increment
in the value of major, which mainly includes: AOT calling conventions, AOT file
format, wasm_export.h, wasm_c_api.h, and so on.
- minor. It represents new features, including MVP/POST-MVP features, libraries,
WAMR private ones, and so one.
- patch. It represents patches.
The new version will start from 1.0.0. Update the help info and version showing for
iwasm and wamrc.
Upgrade `cmake_minimum_required` from `(VERSION 2.8)` to `(VERSION 2.9)` to
yield the warning:
"Compatibility with CMake < 2.8.12 will be removed from a future version of CMake"
Add "-Wno-unused" for CMAKE_CXX_FLAGS to yield the compilation warnings
when build LLVM JIT.
Fix the link error when code coverage is enabled.
Unbreak builds with a bit older nuttx/apps for now.
Note: The default value here (0) was chosen to match the new default
in nuttx/apps, which is different from the old value used before
the introduction of WASM_STACK_GUARD_SIZE. (1024)
Let iwasm return non-zero value when running failed
so that the caller (e.g. test framework) can check the
running status according to the return value.
Add a new option WAMR_BUILD_STACK_GUARD_SIZE to set the custom
stack guard size. For most RTOS systems, we use the native stack base
address as the check boundary which may be not safe as POSIX based
systems (like Linux).
Import WAMR Fast JIT which is a lightweight JIT with quick startup, small footprint,
relatively good performance (~40% to ~50% of LLVM JIT) and good portability.
Platforms supported: Linux, MacOS and Linux SGX.
Arch supported: x86-64.
When WAMR_BUILD_TARGET isn't set, choosing right target is decided
by checking `CMAKE_SIZEOF_VOID_P` variable. However, choosing `X86_32`
target is not doing specifically checking size of void pointer. It is kind
a fallback target for others.
This patch explicitly checks the size of void pointer before setting the target
to `X86_32` to fix the issue.
Implement Berkeley Socket API for Intel SGX
- bring Berkeley socket API in Intel SGX enclaves,
- adapt the documentation of the socket API to mention Intel SGX enclaves,
- adapt _iwasm_ in the mini-product _linux-sgx_ to support the same option as the one for _linux_,
- tested on the socket sample as provided by WAMR (the TCP client/server).
Refer to [Networking API design](https://github.com/WebAssembly/WASI/issues/370)
and [feat(socket): berkeley socket API v2](https://github.com/WebAssembly/WASI/pull/459):
- Support the socket API of synchronous mode, including `socket/bind/listen/accept/send/recv/close/shutdown`,
the asynchronous mode isn't supported yet.
- Support adding `--addr-pool=<pool1,pool2,..>` argument for command line to identify the valid ip address range
- Add socket-api sample and update the document
Since DPFPU depends on FPU, if FPU is enabled we will never enter DPFPU
branch since `ifeq (${CONFIG_ARCH_FPU}, y)` is always true.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Allow compilation on Windows MinGW, see build_wamr.md for more details.
Note that WASI and some other smallish details are still not supported, but
we have a starting point. See more discussion at #993
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup
the launching process:
https://llvm.org/docs/ORCv2.html#laziness
Main modifications:
- Create LLVM module for each wasm function, wrap it with thread safe module
so that the modules can be compiled parallelly
- Lookup function from aot module instance's func_ptrs but not directly call the
function to decouple the module relationship
- Compile the function when it is first called and hasn't been compiled
- Create threads to pre-compile the WASM functions parallelly when loading
- Set Lazy JIT as default, update document and build/test scripts
Refine is_xip_file check, when e_type isn't E_TYPE_XIP, just return false
and no need to go through all the other sections of the AOT file.
Refine pointer range check, convert pointer to uintptr_t type before
comparison to yield possible sanitizer pointer overflow error.
Enable running XIP file on Windows platform.
And add more strict checks for wamrc to report error when the input file
is same with output file, or the input file is AOT file but not wasm file.
Fix some issues on MacOS platform
- Enable libc-wasi by default
- Set target abi to "gnu" if it is not set for wamrc to avoid generating
object file of unsupported Mach-O format
- Set `<vendor>-<sys>` info according to target abi for wamrc to support
generating AOT file for other OSs but not current host
- Set cpu name if arch/abi/cpu are not set to avoid checking SIMD
capability failed
- Set size level to 1 for MacOS/Windows platform to avoid relocation type
unsupported warning
- Clear posix_memmap.c compiling warning
- Fix spec case test script issues, enable test spec cases on MacOS
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Various fixes and beautifications coordinated with @1c3t3a,
fixes 2 of the 3 all remaining issues from #892:
- enable to os_mmap executable memory
- fix os_malloc/os_realloc/os_free issues
- implement os_thread_get_stack_boundary
- add build scripts to include with esp-idf to use wamr as
an ESP-IDF component
- update sample and document
Update the Zephyr document to provide more detailed instructions, and add info
about espressif toolchain too.
Add ESP32C3 (RISC-V) and Particle Argon boards support to the zephyr platform
sample. More boards are possible, but the script doesn't scale well and is to be
improved in future.
Add Dockerfile to kickstart a Zephyr development environment as it can be rather
confusing for new users.
- use platform independent data types in debug-engine library
- add os_socket APIs and provide windows and posix implementation
- avoid using platform related header files in non-platform layer
- use format specifiers macros for sprintf and sscanf
- change thread handle type from uint64 to korp_tid
- add lock when sending socket packet to avoid thread racing
This PR introduces an implementation of the WAMR platform APIs for ESP-IDF and enables support for Espressif microcontrollers, and adds the documentation around how to build WAMR for ESP-IDF.
This PR is related to the following issues at WAMR: closes#883, #628, #449 and #668 as well as [#4735](https://github.com/espressif/esp-idf/issues/4735) at the esp-idf repo. It implements most functions required by [platform_api_vmcore.h](https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/shared/platform/include/platform_api_vmcore.h).
The PR works in interpreter mode on Esp32c3 and Esp32. For the AOT mode, currently errors occur on both platforms with `Guru Meditation Error`. It seems that the AOT code isn't run with shared memory as os_mmap() allocates memory with malloc() API, it is to be fixed in the future.
Refine some codes in wasm loader
Add -Wshadow to gcc compile flags and fix some variable shadowed issues
Fix function parameter/return types not checked issue
Fix fast-interp loader reserve_block_ret() not handle V128 return type issue
Fix mini loader load_table_segment_section() failed issue
Add detailed comments for argc argument in wasm_runtime_call_wasm()
Auto detect whether file is XIP file before loading module in posix like and
linux-sgx platforms, and if yes, mmap executable memory automatically to
run the XIP file.
Add document about XIP feature.
Enable test spec cases with XIP feature.
Applied on the nuttx platform, some memory configurations are added,
and aot and interp can be completely closed or opened, thereby reducing
the flash size.
Zephyr can apply board specific .conf and .overlay automatically,
move .conf to boards to make it easy to support different board without
addition command parameters.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
