Implement 2-level Multi-tier JIT engine: tier-up from Fast JIT to LLVM JIT to
get quick cold startup by Fast JIT and better performance by gradually
switching to LLVM JIT when the LLVM JIT functions are compiled by the
backend threads.
Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1302
When building LLVM, if multiple SDKs are installed in `/Library/Developer/CommandLineTools/SDKs`,
headers from multiple SDKs are picked, generating build errors as describe here:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1758.
Without disabling the bound checks (i.e. -DWAMR_DISABLE_HW_BOUND_CHECK=1)
when building `iwasm`, the `-g=127.0.0.1:1234` parameter makes the runtime crash.
Update the document to avoid the issues.
Signed-off-by: eloparco <eloparco@amazon.com>
Update build wasm app document, add how to set buildflags for Rust
project to reduce the footprint.
Clear Windows warnings and a shadow warning in aot_emit_numberic.c
Add a new options to control the native stack hw bound check feature:
- Besides the original option `cmake -DWAMR_DISABLE_HW_BOUND_CHECK=1/0`,
add a new option `cmake -DWAMR_DISABLE_STACK_HW_BOUND_CHECK=1/0`
- When the linear memory hw bound check is disabled, the stack hw bound check
will be disabled automatically, no matter what the input option is
- When the linear memory hw bound check is enabled, the stack hw bound check
is enabled/disabled according to the value of input option
- Besides the original option `--bounds-checks=1/0`, add a new option
`--stack-bounds-checks=1/0` for wamrc
Refer to: https://github.com/bytecodealliance/wasm-micro-runtime/issues/1677
Use the cmake variable `WAMR_BUILD_GLOBAL_HEAP_POOL` and
`WAMR_BUILD_GLOBAL_HEAP_SIZE` to enable/disable the global heap pool
and set its size. And set the default global heap size in core/config.h and
the cmake files.
As a result, the developers who build iwasm can easily enable/disable the
global heap pool and change its size regardless of the iwasm implementation,
without manually finding and patching the right location for that value.
Refactor LLVM JIT for some purposes:
- To simplify the source code of JIT compilation
- To simplify the JIT modes
- To align with LLVM latest changes
- To prepare for the Multi-tier JIT compilation, refer to #1302
The changes mainly include:
- Remove the MCJIT mode, replace it with ORC JIT eager mode
- Remove the LLVM legacy pass manager (only keep the LLVM new pass manager)
- Change the lazy mode's LLVM module/function binding:
change each function in an individual LLVM module into all functions in a single LLVM module
- Upgraded ORC JIT to ORCv2 JIT to enable lazy compilation
Refer to #1468
This PR integrates an Intel SGX feature called Intel Protection File System Library (IPFS)
into the runtime to create, operate and delete files inside the enclave, while guaranteeing
the confidentiality and integrity of the data persisted. IPFS can be referred to here:
https://www.intel.com/content/www/us/en/developer/articles/technical/overview-of-intel-protected-file-system-library-using-software-guard-extensions.html
Introduce a cmake variable `WAMR_BUILD_SGX_IPFS`, when enabled, the files interaction
API of WASI will leverage IPFS, instead of the regular POSIX OCALLs. The implementation
has been written with light changes to sgx platform layer, so all the security aspects
WAMR relies on are conserved.
In addition to this integration, the following changes have been made:
- The CI workflow has been adapted to test the compilation of the runtime and sample
with the flag `WAMR_BUILD_SGX_IPFS` set to true
- Introduction of a new sample that demonstrates the interaction of the files (called `file`),
- Documentation of this new feature
Implement more socket APIs, refer to #1336 and below PRs:
- Implement wasi_addr_resolve function (#1319)
- Fix socket-api byte order issue when host/network order are the same (#1327)
- Enhance sock_addr_local syscall (#1320)
- Implement sock_addr_remote syscall (#1360)
- Add support for IPv6 in WAMR (#1411)
- Implement ns lookup allowlist (#1420)
- Implement sock_send_to and sock_recv_from system calls (#1457)
- Added http downloader and multicast socket options (#1467)
- Fix `bind()` calls to receive the correct size of `sockaddr` structure (#1490)
- Assert on correct parameters (#1505)
- Copy only received bytes from socket recv buffer into the app buffer (#1497)
Co-authored-by: Marcin Kolny <mkolny@amazon.com>
Co-authored-by: Marcin Kolny <marcin.kolny@gmail.com>
Co-authored-by: Callum Macmillan <callumimacmillan@gmail.com>
Use the semantic versioning (https://semver.org) to replace the current date
versioning system, which is more general and is requested by some developers,
e.g. issue #1357.
There are three parts in the new version string:
- major. Any incompatible modification on ABIs and APIs will lead to an increment
in the value of major, which mainly includes: AOT calling conventions, AOT file
format, wasm_export.h, wasm_c_api.h, and so on.
- minor. It represents new features, including MVP/POST-MVP features, libraries,
WAMR private ones, and so one.
- patch. It represents patches.
The new version will start from 1.0.0. Update the help info and version showing for
iwasm and wamrc.
Add a new option WAMR_BUILD_STACK_GUARD_SIZE to set the custom
stack guard size. For most RTOS systems, we use the native stack base
address as the check boundary which may be not safe as POSIX based
systems (like Linux).
Lookup table for i32.const and i64.const for xtensa XIP
Lookup const offset from table for load/store opcodes for xtensa XIP
Fill capability flags for xtensa XIP
Enable lower switch pass for xtensa XIP
Import WAMR Fast JIT which is a lightweight JIT with quick startup, small footprint,
relatively good performance (~40% to ~50% of LLVM JIT) and good portability.
Platforms supported: Linux, MacOS and Linux SGX.
Arch supported: x86-64.
Fix build script to enable hw bound check for interpreter when
AOT is disabled, so as to enable spec cases test for interp with
hw bound check. And fix the issues found.
Support integrating 3rd-party toolchain llc compiler or asm compiler
into wamrc by setting environment variable WAMRC_LLC_COMPILER
or WAMRC_ASM_COMPILER, wamrc will use these tools to generate
object file from LLVM IR firstly, and then refactor the object file into
aot file.
Implement Berkeley Socket API for Intel SGX
- bring Berkeley socket API in Intel SGX enclaves,
- adapt the documentation of the socket API to mention Intel SGX enclaves,
- adapt _iwasm_ in the mini-product _linux-sgx_ to support the same option as the one for _linux_,
- tested on the socket sample as provided by WAMR (the TCP client/server).
Refer to [Networking API design](https://github.com/WebAssembly/WASI/issues/370)
and [feat(socket): berkeley socket API v2](https://github.com/WebAssembly/WASI/pull/459):
- Support the socket API of synchronous mode, including `socket/bind/listen/accept/send/recv/close/shutdown`,
the asynchronous mode isn't supported yet.
- Support adding `--addr-pool=<pool1,pool2,..>` argument for command line to identify the valid ip address range
- Add socket-api sample and update the document
The littlevgl library had changed its name and domain to [LVGL](https://lvgl.io).
See https://blog.lvgl.io/2020-06-01/announcement
We change some names and links accordingly.
Also remove the cloning for tlsf library as it isn't used now.
Allow compilation on Windows MinGW, see build_wamr.md for more details.
Note that WASI and some other smallish details are still not supported, but
we have a starting point. See more discussion at #993
Implement pthread_cond_broadcast wrapper for lib-pthread
- support pthread_cond_broadcast wrapper for posix/linux-sgx/windows
- update document for building multi-thread wasm app with emcc
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup
the launching process:
https://llvm.org/docs/ORCv2.html#laziness
Main modifications:
- Create LLVM module for each wasm function, wrap it with thread safe module
so that the modules can be compiled parallelly
- Lookup function from aot module instance's func_ptrs but not directly call the
function to decouple the module relationship
- Compile the function when it is first called and hasn't been compiled
- Create threads to pre-compile the WASM functions parallelly when loading
- Set Lazy JIT as default, update document and build/test scripts
Currently when calling wasm_runtime_call_wasm() to invoke wasm function
with externref type argument from runtime embedder, developer needs to
use wasm_externref_obj2ref() to convert externref obj into an internal ref
index firstly, which is not convenient to developer.
To align with GC feature in which all the references passed to
wasm_runtime_call_wasm() can be object pointers directly, we change the
interface of wasm_runtime_call_wasm() to allow to pass object pointer
directly for the externref argument, and refactor the related codes, update
the related samples and the document.
Update the Zephyr document to provide more detailed instructions, and add info
about espressif toolchain too.
Add ESP32C3 (RISC-V) and Particle Argon boards support to the zephyr platform
sample. More boards are possible, but the script doesn't scale well and is to be
improved in future.
Add Dockerfile to kickstart a Zephyr development environment as it can be rather
confusing for new users.
This PR introduces an implementation of the WAMR platform APIs for ESP-IDF and enables support for Espressif microcontrollers, and adds the documentation around how to build WAMR for ESP-IDF.
This PR is related to the following issues at WAMR: closes#883, #628, #449 and #668 as well as [#4735](https://github.com/espressif/esp-idf/issues/4735) at the esp-idf repo. It implements most functions required by [platform_api_vmcore.h](https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/shared/platform/include/platform_api_vmcore.h).
The PR works in interpreter mode on Esp32c3 and Esp32. For the AOT mode, currently errors occur on both platforms with `Guru Meditation Error`. It seems that the AOT code isn't run with shared memory as os_mmap() allocates memory with malloc() API, it is to be fixed in the future.
Auto detect whether file is XIP file before loading module in posix like and
linux-sgx platforms, and if yes, mmap executable memory automatically to
run the XIP file.
Add document about XIP feature.
Enable test spec cases with XIP feature.
- move the wait_cond from exec_env to debug_instance, so the debug thread can be waken up by any threads
- process more general query message from debugger
- refine debug instance create/destroy mechanism
- avoid creating debug instance during module instantiating
- avoid blocking execution thread during creating debug instance
- update related documents
Enable ref types feature and bulk memory feature by default for wamrc
and provide "--disable-ref-types", "--disable-bulk-memory" to disable
them.
And remove the ref_type_flag option in wasm_loader.c which is used to
control whether to enable ref types or not when ENABLE_REF_TYPES
macro is enabled in wamrc. As the wasm binary format with ref types
is compatible with the binary format before, we can remove the option.
Also update the spec test scripts.
Enable emitting custom name section to aot file when adding
`--enable-dump-call-stack` or `--enable-dump-call-stack` to
wamrc and there is custom name section in wasm file, which
can be generated by wasi-sdk/emcc "-g" option. So aot runtime
can also get the function name from the custom name section
instead of export section, to which developer should use
`--export-all` for wasi-sdk/emcc to generate export function
names.
Apply clang-format for C source files in folder core/app-mgr,
core/app-framework, and test-tools.
And rename folder component_test to component-test, update
zephyr build document.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Zephyr can apply board specific .conf and .overlay automatically,
move .conf to boards to make it easy to support different board without
addition command parameters.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Apply clang-format for core/iwasm/include, core/iwasm/common and
core/iwasm/aot files.
Update spec cases test script:
- Checkout latest commit of https://github.com/WebAssembly/spec
- Checkout main branch but not master of https://github.com/WebAssembly/threads
- Update wabt to latest version
And update source debugging document.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement source debugging feature for classic interpreter and AOT:
- use `cmake -DWAMR_BUILD_DEBUG_INTERP=1` to enable interpreter debugging
- use `cmake -DWAMR_BUILD_DEBUG_AOT=1` to enable AOT debugging
See doc/source_debugging.md for more details.
Fix document issues: add ARC to supported targets, fix how to build wamrc for MacOS.
Fix spec case test script issue: the latest wabt has enabled simd by default, no need to
add "--enable-simd" option for test script.
Fix simd LLVM IR compilation issue: using index calculated by opcode to access array
element should not be out of array boundary, add bh_assert() for it.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement the latest SIMD opcodes and update LLVM 13.0,
update the llvm build scripts, update the sample workloads‘ build scripts,
and build customized wasi-sdk to build some workloads.
Also refine the CI rules.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Update CI rules to improve CI build speed, update document about arch support and update document of wamrc help.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
And fix libc-builtin print float issue
Add a list to track all third party components
Fix compile error when MEMORY_TRACING is enabled
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement Windows thread/mutex/cond related APIs to support Windows multi-thread feature
Change Windows HW boundary check implementation for multi-thread: change SEH to VEH
Fix wasm-c-api issue of getting AOTFunctionInstance by index, fix wasm-c-api compile warnings
Enable to build invokeNative_general.c with cmake variable
Fix several issues in lib-pthread
Disable two LLVM passes in multi-thread mode to reserve volatile semantic
Update docker script and document to build iwasm with Docker image
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement spec reference-types proposal for interpreter, AOT and JIT, update documents and add sample. And upgrade AOT_CURRENT_VERSION to 3 as AOT file format and AOT module instance layout are changed.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Upgrade it as the latest version of wasi-sdk and emsdk are based on llvm-11, align to it to use the same compiler version. And this also fixes compilation error when building wamrc with llvm-10 in Windows platform.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable to use BH_VPRINTF macro for platform Linux/Windows/Darwin/VxWorks to redirect the stdout output from platform os_printf/os_vprintf, or the wasi output from wasm app to the vprintf like callback function specified by BH_VPRINTF macro of cmake WAMR_BH_VPRINTF variable.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable SIMD by default for wamrc on x86-64 target, for iwasm on platform Linux and Darwin. And update build wamr document, fix app manager compile warning.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
And remove redundant FAST_INTERP macros in wasm_interp_fast.c, and fix wamrc --help wrong line order issue.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Update wasm app build scripts for wasi-sdk-12.0: add --export=__main_argc_argv, remove --no-threads
Lookup function with name "__main_argc_argv" as main function besides "main"
Change module_malloc to runtime_malloc in wasi native lib
Refine classic interpreter op_block and op_br_table
Refine faster interpreter op_br_table
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
The global doc/linux_sgx.md needs to explicitly describe the methods to
build a debug enclave and hardware running mode. Because using debug key
to signing enclave image rather than production key is still not trivial
in reality.
For the adaption of Inclavare Containers part, add a prolog and give
more details in order to enhance the readability.
Signed-off-by: Jia Zhang <zhang.jia@linux.alibaba.com>
This commit mainly simplifies the description about building a
debug and hw mode enclave.
Signed-off-by: Jia Zhang <zhang.jia@linux.alibaba.com>
Co-authored-by: root <root@rs1g04412.et2sqa>
Add the function name field for internal function struct
Signed-off-by: Zhongmin Wu <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Co-authored-by: Zhongmin Wu <vwzm@live.com>
