Refine some codes in wasm loader
Add -Wshadow to gcc compile flags and fix some variable shadowed issues
Fix function parameter/return types not checked issue
Fix fast-interp loader reserve_block_ret() not handle V128 return type issue
Fix mini loader load_table_segment_section() failed issue
Add detailed comments for argc argument in wasm_runtime_call_wasm()
Treat R_AARCH64_JUMP26 same as R_AARCH64_CALL26, both of their
relocation operation is S + A - P, the difference is that one is for call
(BL) instructions, the other is for jump (B) instructions.
Auto detect whether file is XIP file before loading module in posix like and
linux-sgx platforms, and if yes, mmap executable memory automatically to
run the XIP file.
Add document about XIP feature.
Enable test spec cases with XIP feature.
- move the wait_cond from exec_env to debug_instance, so the debug thread can be waken up by any threads
- process more general query message from debugger
- refine debug instance create/destroy mechanism
- avoid creating debug instance during module instantiating
- avoid blocking execution thread during creating debug instance
- update related documents
Don't throw exception when module_malloc memory failed:
- Exception will terminate the wasm app, it's not necessary since app can
check the result of dynamic allocation and do some cleanup or fallback
operation on failure instead of 'crash' directly.
- In acquire_wait_info, call hasn_map_find only when the address isn't NULL,
or there are many senseless error logs
When auxiliary stack global isn't found in wasm app, it must be unused in
the wasm app, we set it to __heap_base global and set its size to 0, so as to
shrink the linear memory to reduce the memory consumption.
The debug_instance_list is not protected by mutex, and the debug instance uses
the length of this list as its id. If several threads create debug instance at the
same time, they may get duplicated/same ids. This patch adds mutex to protect
this list.
The destroy_callback_list variable in thread_manager_destroy is not cleared.
When wamr is started as a task, after exiting and restarting, it will loop
endlessly in traverse_list.
Signed-off-by: zhouliang3 <zhouliang3@xiaomi.com>
Enable ref types feature and bulk memory feature by default for wamrc
and provide "--disable-ref-types", "--disable-bulk-memory" to disable
them.
And remove the ref_type_flag option in wasm_loader.c which is used to
control whether to enable ref types or not when ENABLE_REF_TYPES
macro is enabled in wamrc. As the wasm binary format with ref types
is compatible with the binary format before, we can remove the option.
Also update the spec test scripts.
Fix some potential pointer overflows in aot applying relocations and
several other places.
And add sanitizer compiler flags to wamrc CMakeLists.txt to detect
such issues.
Enable LLVM link time optimization for AOT and enable it by default,
and provide "wamrc --disable-llvm-lto" option to disable it.
And modify the spec test script, disable it by default when testing
spec cases.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
This patch enables huge page support for posix platforms for performance
reason, if the request size to mmap is larger than 2MB, then we use madvise
to set some pages to huge page.
And add macro control to enable tracing the mmap/munmap.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Fix fail to get builtin functions in indirect call (XIP) mode, e.g.
function aot_alloc_frame, aot_memory_init, aot_table_init and so on.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Currently the string in the wasm/aot file will be duplicated and stored
into const string list/set in interpreter/aot loader, which leads to extra
unnecessary memory consumption if the file buffer can be referred to
after loading. We refine the string storage by:
- if the file buffer can be referred to after loading and it is writable, we
reuse the file buffer to store the string but not store it into the const
string set: move string backward and append '\0'
- emit string with '\0' only for XIP mode in which the AOT file is readonly
- if the file buffer cannot be referred to, e.g. in app manager, keep the
same behavior as before
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
For cmake based project, there are default values in .cmake,
but for other build system, may these symbols were not defined in their
build system and warnings (XXX is not defined) were generated while building.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Enable emitting custom name section to aot file when adding
`--enable-dump-call-stack` or `--enable-dump-call-stack` to
wamrc and there is custom name section in wasm file, which
can be generated by wasi-sdk/emcc "-g" option. So aot runtime
can also get the function name from the custom name section
instead of export section, to which developer should use
`--export-all` for wasi-sdk/emcc to generate export function
names.
Apply clang-format for C source files in folder core/app-mgr,
core/app-framework, and test-tools.
And rename folder component_test to component-test, update
zephyr build document.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
The size used to apply "size = align_uint(size, 4)" may be different
when calculating total size (in get_object_data_sections_size) and
emitting actual data (in aot_emit_object_data_section_info) for the
object data section.
This patch fixes the "Error: emit object data section info failed".
Apply clang-format for core/iwasm/compilation and core/iwasm/libraries files.
Add wasm-c-api empty_imports sample to workflow test.
And enable triggering workflow when core/config.h changes.
Allow empty import array (but not NULL) for wasm-c-api wasm_instance_new,
which makes the code more robust, especially in use cases where imports
might be constructed dynamically.
And add a test case in samples/wasm-c-api/src/empty_imports.c to test it.
- fix data race issue between debug control thread and main thread
- fix possible memory leaks in breakpoints list
- fix memory uninitialized issues
- remove unused data structures
- add more checks when handling packet and args
- fix mini-loader issues
- fix config_common.cmake fast interp prompt issue
Apply clang-format for core/iwasm/include, core/iwasm/common and
core/iwasm/aot files.
Update spec cases test script:
- Checkout latest commit of https://github.com/WebAssembly/spec
- Checkout main branch but not master of https://github.com/WebAssembly/threads
- Update wabt to latest version
And update source debugging document.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement source debugging feature for classic interpreter and AOT:
- use `cmake -DWAMR_BUILD_DEBUG_INTERP=1` to enable interpreter debugging
- use `cmake -DWAMR_BUILD_DEBUG_AOT=1` to enable AOT debugging
See doc/source_debugging.md for more details.
Fix document issues: add ARC to supported targets, fix how to build wamrc for MacOS.
Fix spec case test script issue: the latest wabt has enabled simd by default, no need to
add "--enable-simd" option for test script.
Fix simd LLVM IR compilation issue: using index calculated by opcode to access array
element should not be out of array boundary, add bh_assert() for it.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement the latest SIMD opcodes and update LLVM 13.0,
update the llvm build scripts, update the sample workloads‘ build scripts,
and build customized wasi-sdk to build some workloads.
Also refine the CI rules.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
The WASM C API now requires the use of vector types in certain apis.
Switching WAMR to use the new call signatures improves "drop in"
compilation compatibility between WAMR and other implementations
from a C-api embedding program's perspective.
* wasm_func_callback_t type has been updated to use wasm_val_vec_t
* wasm_func_callback_with_env_t type has been updated to use wasm_val_vec_t
* wasm_func_call() has been updated to use wasm_val_vec_t
* wasm_instance_new() has been updated to use wasm_extern_vec_t*
* wasm_instance_new_with_args() has been updated to use wasm_extern_vec_t*
* wasm_runtime_invoke_c_api_native() has been updated to support vector types
in native callbacks without modifying the contract with the interpreter code.
* All users of the modified functions (including samples/wasm-c-api/src/*.c)
have been appropriately updated.
Before calling import function of sub module inst in multi-module mode,
we should copy the arguments to output area by using the func inst of
sub module inst but not func inst of current module inst.
Some of the function arguments were named as "export" in core/iwasm/common/wasm_runtime_common.h, which result in conflicts with the C++11 keyword. Change them to other to fix the compilation error.
The last argument arg of pthread_create passed from wasm app is opaque
for the library and host, it should be returned back to the wasm app without
any interpretation. There is no point to translate it to a native address.
Actually it harms because the validation seems broken.
Implement XIP (Execution In Place) feature for AOT mode to enable running the AOT code inside AOT file directly, without memory mapping the executable memory for AOT code and applying relocations for text section. Developer can use wamrc with "--enable-indirect-mode --disable-llvm-intrinsics" flags to generate the AOT file and run iwasm with "--xip" flag. Known issues: there might still be some relocations in the text section which access the ".rodata" like sections.
And also enable ARC target support for both interpreter mode and AOT mode.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement wasm_runtime_init_thread_env() for Windows platform by calling os_thread_env_init(): if current thread is created by developer himself but not runtime, developer should call wasm_runtime_init_thread_env() to init the thread environment before calling wasm function, and call wasm_runtime_destroy_thread_env() before thread exits.
And clear compile warnings for Windows platform, fix compile error for AliOS-Things platform
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
And output detail info when install wasm app failed, update document and fix some compile warnings and errors.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable to build wamrc with custom llvm, enable to auto detect processor on apple silicon, and fix some compile warnings.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Enable RISCV AOT support, the supported ABIs are LP64 and LP64D for riscv64, ILP32 and ILP32D for riscv32.
For wamrc:
use --target=riscv64/riscv32 to specify the target arch of output AOT file,
use --target-abi=lp64d/lp64/ilp32d/ilp32 to specify the target ABI,
if --target-abi isn't specified, by default lp64d is used for riscv64, and ilp32d is used for riscv32.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
Co-authored-by: wenyongh <wenyong.huang@intel.com>
And add wasm_engine_new_with_args() declaration in wasm_c_api.h
Fix wasm-c-api frame func_offset issue in fast interp mode
Remove sanitize compiler flag in product-mini linux CMakeLists.txt
And enable to cache compiled AOT file buffer for wasm-c-api JIT mode
Avoid checks that rely on undefined C behavior
Fix issues of wasm-c-api sample trap and callback_chain
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
And enable to copy back the return value of wasm main function when calling wasm_application_execute_main, add license headers in wasm-c-api samples, fix several issues reported by klocwork.
And fix libc-builtin print float issue
Add a list to track all third party components
Fix compile error when MEMORY_TRACING is enabled
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement Windows thread/mutex/cond related APIs to support Windows multi-thread feature
Change Windows HW boundary check implementation for multi-thread: change SEH to VEH
Fix wasm-c-api issue of getting AOTFunctionInstance by index, fix wasm-c-api compile warnings
Enable to build invokeNative_general.c with cmake variable
Fix several issues in lib-pthread
Disable two LLVM passes in multi-thread mode to reserve volatile semantic
Update docker script and document to build iwasm with Docker image
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Implement more wasm-c-api APIs to support Envoy integration:
- sync up with latest c-api definition
- change CMakeLists to export necessary headers and install the static library of iwasm
- enable to export tables and memories
- support memorytype and tabletype APIs
- update wasm-c-api sampels
- enable to export importtype APIs
And refine bazel scripts for sample XNNPACK workload, add license headers for sample simple.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Fixed a typo in aot_reloc_aarch64.c.
Remove -m32 option in host tool CMakeLists.txt and change data type of structure fields to make host tool work in 64-bit.
Signed-off-by: Bao Haojun <baohaojun@lixiang.com>
Implement spec reference-types proposal for interpreter, AOT and JIT, update documents and add sample. And upgrade AOT_CURRENT_VERSION to 3 as AOT file format and AOT module instance layout are changed.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Some libc APIs required by wasi native lib are missing in some Android API versions, only when the version >= 24, all APIs are supported. Add the missing APIs in android platform layer, leave them empty, report error and return failed if they are called. Also update CMakeLists.txt to enable libc wasi by default.
Co-authored-by: Wenyong Huang <wenyong.huang@intel.com>
Fix sample littlevgl/guil build issues on zephyr platform, modify code of zephyr platform by adding some macros to control some apis to fit different zephyr version, and align to zephyr-v2.3.0 for sample littlevgl/gui as the latest zephyr version (>=v2.4.0) requires that the thread stack for the thread to create must be defined by macro but not allocating stack memory dynamically.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
AssemblyScript's latest compiler 0.18 exports __new__/__pin__/unpin APIs to allocate/retain/free memory instead of __new/__retain/__release APIs in older version, we lookup functions of both version to make it compatible for both version.
Implement Windows PE file relocation type IMAGE_REL_AMD64_ADDR64/ADDR32/REL32, implement relocation for symbol "__xmm@xxx"/"__plt@xxx"/".rdata", implement Windows invokeNative simd asm code and enable SIMD by default for windows platform. Also update wamrc tool.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Remove check for heap_size==0 when creating wasi ctx, as the related data structures are allocated from global heap instead of app heap now, so it also works when app heap isn't created.
Also add v128 type for Windows so as to fix wamrc compilation error in Windows platform.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
The native function might call wasm function exported, in which the memory.grow opcode might be executed, and interpreter should update memory size after that, or load/store opcodes may run failed with "out of bounds memory access" exception thrown.
Update tensorflow sample patch, allow tensorflow wasm app to grow memory so as to run more models. And fix some compile issues of littlevgl zephyr sample for latest zephyr source code.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable to use BH_VPRINTF macro for platform Linux/Windows/Darwin/VxWorks to redirect the stdout output from platform os_printf/os_vprintf, or the wasi output from wasm app to the vprintf like callback function specified by BH_VPRINTF macro of cmake WAMR_BH_VPRINTF variable.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Enable SIMD by default for wamrc on x86-64 target, for iwasm on platform Linux and Darwin. And update build wamr document, fix app manager compile warning.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Remove the limit of app file size no larger than 1 MB, fix possible memory leak issues when fail to install app file, change message size of aee_host_msg_callback() from uint16 type to uint32 type to fix possible integer overflow issue, and fix some coding style issues of app manager.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
The total size actually allocated by ealloc_hmu() might be larger than the size required to allocate, we reset the total size after allocating, so that if heap verification feature is enabled, the data to verify can be written to the right place of the suffix verification area. And in gc_realloc_vo_internal(), free the heap lock until the original data is copied to new object to return.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
And add native symbol node to list head when registering native symbol, so as to lookup developer's registered node firstly when linking.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
In some platforms, allocating memory with size 0 may return NULL but not an empty memory block, which causes runtime load, instantiate or execute wasm/aot file failed. We add checks to try to avoid allocating memory in runtime if the size is 0. And in wasm_runtime_malloc/free, output warning if allocate memory with size 0 and free memory with NULL ptr.
Also fix some coding style issues, fix handle riscv32 ilp32d issue, and fix several wasm-c-api issues.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
And remove redundant FAST_INTERP macros in wasm_interp_fast.c, and fix wamrc --help wrong line order issue.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Re-implement aot call_indirect opcode translation: when calling non-import function, translate it by LLVM call IR to call the function in AOTed code, so as to avoid calling runtime aot_call_indirect API which is much slower. For import function, keep calling aot_call_indirect API due to the possible pointer/string argument conversion.
And add prompt info while app heap is corrupted, change emit_leb to emit_uint32 inter fast-interp to refine footprint.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Modify the macro control sentences as darwin platform doesn't support mmap with MAP_32BIT flag to put the mapping address into 0-2G range.
Co-authored-by: Wenyong Huang <wenyong.huang@intel.com>
Update the build scripts of sample workloads (meshoptimizer/bwa/wasm-av1) to use the wasi-sdk-12.0 firstly to build the workload and discard clang-11, as wasi-sdk-12 supports wasi, simd and pthread better. And update the related documents.
Also modify wasm mini loader to sync up with the change of wasm normal loader.
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Update wasm app build scripts for wasi-sdk-12.0: add --export=__main_argc_argv, remove --no-threads
Lookup function with name "__main_argc_argv" as main function besides "main"
Change module_malloc to runtime_malloc in wasi native lib
Refine classic interpreter op_block and op_br_table
Refine faster interpreter op_br_table
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Fix potential memory leak issue when using llvm::EngineBuilder().selectTarget()
Fix issue of accessing aot_value's fields after it is freed
Fix JIT not print failed to link import warning
Change some error messages: 'fail to' to 'failed to'
Update error message when SIMD isn't enabled
Fix install littlevgl wasm app of wasi version failed
Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Motivation:
we found some issues during enable the WAMR on arm devices, such as relocation issues, stack alignment issues.
Proposed change:
We optimized the relocation process for arm platform, for relocation jump table, the SP should be aligned to 16 bytes. And we also make "getentropy" worked for other non-supported platform.
add more checks to enhance security
clear "wasi proc exit" exception before return to caller in wasm/aot call functions
fix memory profiling issue
change movdqa to movdqu in simd invokeNative asm codes to fix issue of unaligned address access
move setjmp/longjmp from libc-builtin to libc-emcc
fix zephyr platform compilation issue in latest zephyr version
* Add the tail-call feature support for classic-interp
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
* add CI for tail call and custom name section
* add CI for tail call and custom name section
* update CI for mac
Co-authored-by: Xu Jun <693788454@qq.com>
Add the function name field for internal function struct
Signed-off-by: Zhongmin Wu <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Co-authored-by: Zhongmin Wu <vwzm@live.com>
Add the Windows COFF format support to wamr-compiler and iwasm can
load and excute it on Windows(X64) platform.
Signed-off-by: Wu Zhongmin <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Co-authored-by: Wu Zhongmin <vwzm@live.com>
* Add Windows support for C-API and Runtime API libraries and examples.
Signed-off-by: Wu Zhongmin <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
* Address the review comments
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
* Rewrite the the bh_getopt to make it avaliable for more kinds of options
Signed-off-by: Wu Zhongmin <vwzm@live.com>
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
* Add the license header
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Co-authored-by: Zhongmin Wu <vwzm@live.com>
Fix the build issues on mac for basic/multi-module/multi-thread/
simple/spawn-thread/wasm-c-api under samples.
And all these samples could be run as expected.
Signed-off-by: Xiaokang Qin <xiaokang.qxk@antgroup.com>
Also implement native stack overflow check with hardware trap for 64-bit platforms
Refine classic interpreter and fast interpreter to improve performance
Update document
Update all links accordingly. Also update links to other repositories
whose branches have renamed.
The references to repositories whose branches have not renamed should be
referencing specific commits anyway, so reference those specific commits
by hash.
* Missing SGX SDK Include fixed
* Update shared_platform.cmake
* CMakeFile remove stdlib from untrusted part
* Added two times in function description zero as possible return value
* Update shared_platform.cmake
Co-authored-by: Joshua Heinemann <heineman@ibr.cs.tu-bs.de>
Co-authored-by: wenyongh <wenyong.huang@intel.com>
* add darwin support for wamr-compiler compiling
* add darwin support for wamr-compiler
* Update CMakeLists.txt
* MacOs: fix wamrc Error invaid llvm binary bin_type
Co-authored-by: wenyongh <wenyong.huang@intel.com>
