Commit Graph

87 Commits

Author SHA1 Message Date
zoraaver
fa5e9d72b0
Abstract POSIX filesystem functions (#2585)
To allow non-POSIX platforms such as Windows to support WASI libc
filesystem functionality, create a set of wrapper functions which provide a
platform-agnostic interface to interact with the host filesystem. For now,
the Windows implementation is stubbed but this will be implemented
properly in a future PR. There are no functional changes in this change,
just a reorganization of code to move any direct POSIX references out of
posix.c in the libc implementation into posix_file.c under the shared
POSIX sources.

See https://github.com/bytecodealliance/wasm-micro-runtime/issues/2495 for a
more detailed overview of the plan to port the WASI libc filesystem to Windows.
2023-10-19 19:19:39 +08:00
YAMAMOTO Takashi
6c846acc59
Implement module instance context APIs (#2436)
Introduce module instance context APIs which can set one or more contexts created
by the embedder for a wasm module instance:
```C
    wasm_runtime_create_context_key
    wasm_runtime_destroy_context_key
    wasm_runtime_set_context
    wasm_runtime_set_context_spread
    wasm_runtime_get_context
```

And make libc-wasi use it and set wasi context as the first context bound to the wasm
module instance.

Also add samples.

Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/2460.
2023-09-07 14:54:11 +08:00
TianlongLiang
c820643b2b
wamrc: More friendly to print help info (#2451)
Allow wamrc to print help info like below:
```bash
wamrc --target=help
wamrc --target-abi=help
wamrc --target=<target> --cpu=help
wamrc --target=<target> --cpu=help --cpu-features=+help
```
2023-08-16 11:28:45 +08:00
Florian Castellane
f34e7431fd
Fix typos in wamrc print_help() (#2442) 2023-08-09 14:31:02 +08:00
Huang Qi
aafea39b8c
Add "--enable-builtin-intrinsics=<flags>" option to wamrc (#2341)
Refer to doc/xip.md for details.
2023-07-06 18:20:35 +08:00
Huang Qi
6a310cc88a
Add "--native-lib=<lib>" option to wamrc (#2342) 2023-07-06 12:25:28 +08:00
Huang Qi
44f4b4f062
Add "--enable-llvm-passes=<passes>" option to wamrc (#2335)
Add "--enable-llvm-passes=<passes>" option to wamrc for customizing LLVM passes
2023-07-04 12:20:52 +08:00
Huang Qi
df98d9fb4a
Add "--xip" option for wamrc (#2336)
Add shorthand "--xip" option for wamrc, which is equal to
"--enalbe-indirect-mode --disable-llvm-intrinsics"
2023-07-04 12:03:40 +08:00
YAMAMOTO Takashi
ce6d1fd8fe
wamrc: Add an incompatibility note in the help message (#2276)
An example of such optimizations:
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1752
2023-06-09 08:12:34 +08:00
YAMAMOTO Takashi
6e3c3fe9ec
Fix build error with LLVM 16 (#2259) 2023-06-06 13:45:18 +08:00
Wenyong Huang
8d88471c46
Implement AOT static PGO (#2243)
LLVM PGO (Profile-Guided Optimization) allows the compiler to better optimize code
for how it actually runs. This PR implements the AOT static PGO, and is tested on
Linux x86-64 and x86-32. The basic steps are:

1. Use `wamrc --enable-llvm-pgo -o <aot_file_of_pgo> <wasm_file>`
   to generate an instrumented aot file.
2. Compile iwasm with `cmake -DWAMR_BUILD_STATIC_PGO=1` and run
      `iwasm --gen-prof-file=<raw_profile_file> <aot_file_of_pgo>`
    to generate the raw profile file.
3. Run `llvm-profdata merge -output=<profile_file> <raw_profile_file>`
    to merge the raw profile file into the profile file.
4. Run `wamrc --use-prof-file=<profile_file> -o <aot_file> <wasm_file>`
    to generate the optimized aot file.
5. Run the optimized aot_file: `iwasm <aot_file>`.

The test scripts are also added for each benchmark, run `test_pgo.sh` under
each benchmark's folder to test the AOT static pgo.
2023-06-05 09:17:39 +08:00
Wenyong Huang
76be848ec3
Implement the segue optimization for LLVM AOT/JIT (#2230)
Segue is an optimization technology which uses x86 segment register to store
the WebAssembly linear memory base address, so as to remove most of the cost
of SFI (Software-based Fault Isolation) base addition and free up a general
purpose register, by this way it may:
- Improve the performance of JIT/AOT
- Reduce the footprint of JIT/AOT, the JIT/AOT code generated is smaller
- Reduce the compilation time of JIT/AOT

This PR uses the x86-64 GS segment register to apply the optimization, currently
it supports linux and linux-sgx platforms on x86-64 target. By default it is disabled,
developer can use the option below to enable it for wamrc and iwasm(with LLVM
JIT enabled):
```bash
wamrc --enable-segue=[<flags>] -o output_file wasm_file
iwasm --enable-segue=[<flags>] wasm_file [args...]
```
`flags` can be:
    i32.load, i64.load, f32.load, f64.load, v128.load,
    i32.store, i64.store, f32.store, f64.store, v128.store
Use comma to separate them, e.g. `--enable-segue=i32.load,i64.store`,
and `--enable-segue` means all flags are added.

Acknowledgement:
Many thanks to Intel Labs, UC San Diego and UT Austin teams for introducing this
technology and the great support and guidance!

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
Co-authored-by: Vahldiek-oberwagner, Anjo Lucas <anjo.lucas.vahldiek-oberwagner@intel.com>
2023-05-26 10:13:33 +08:00
TianlongLiang
7f5d9d4f5e
Update document for iwasm/wamrc dependent packages (#2183) 2023-05-08 16:30:56 +08:00
YAMAMOTO Takashi
2b896c80ef
wamrc: Add --stack-usage option (#2158) 2023-04-28 13:56:44 +08:00
liang.he
d0fb2716b6
Fix several issues in CI binary releasing (#2064) 2023-03-26 12:19:45 +08:00
Wang Xin
c2b73eabe2
Readme refactoring (#2038) 2023-03-19 08:05:57 +08:00
Wenyong Huang
9b9ae0cfac
Update cmake files and wamr-test-suites to support collect code coverage (#1992)
Support collecting code coverage with wamr-test-suites script by using
lcov and genhtml tools, eg.:
  cd tests/wamr-test-suites
  ./test_wamr.sh -s spec -b -P -C

The default code coverage and html files are generated at:
  tests/wamr-test-suites/workspace/wamr.lcov
  tests/wamr-test-suites/workspace/wamr-lcov.zip

And update wamr-test-suites scripts to support testing GC spec cases to
avoid frequent synchronization conflicts between branch main and dev/gc.
2023-02-28 17:38:18 +08:00
Wenyong Huang
e170c355a2 Merge branch main into dev/wasi_threads 2023-02-17 08:46:12 +08:00
YAMAMOTO Takashi
aab875b91f
wamr-compiler: enable wasi-threads (#1938)
cf. https://github.com/bytecodealliance/wasm-micro-runtime/pull/1893
2023-02-07 12:34:33 +08:00
YAMAMOTO Takashi
7d3b2a8773
Make memory profiling show native stack usage (#1917) 2023-02-01 11:52:15 +08:00
Wenyong Huang
fbb369473d Fix link cxx object file error when building wamrc (#1794) 2022-12-20 14:37:07 +08:00
Wenyong Huang
aa6a7a2069
Fix link cxx object file error when building wamrc (#1794) 2022-12-08 18:14:02 +08:00
Wenyong Huang
7fd37190e8
Add control for the native stack check with hardware trap (#1682)
Add a new options to control the native stack hw bound check feature:
- Besides the original option `cmake -DWAMR_DISABLE_HW_BOUND_CHECK=1/0`,
  add a new option `cmake -DWAMR_DISABLE_STACK_HW_BOUND_CHECK=1/0`
- When the linear memory hw bound check is disabled, the stack hw bound check
   will be disabled automatically, no matter what the input option is
- When the linear memory hw bound check is enabled, the stack hw bound check
  is enabled/disabled according to the value of input option
- Besides the original option `--bounds-checks=1/0`, add a new option
  `--stack-bounds-checks=1/0` for wamrc

Refer to: https://github.com/bytecodealliance/wasm-micro-runtime/issues/1677
2022-11-07 18:26:33 +08:00
YAMAMOTO Takashi
654ac5feca
Use cmake POSITION_INDEPENDENT_CODE instead of hardcoding -pie -fPIE (#1598)
This fixes unused option warnings on -pie for macOS.
(On macOS cmake produces "-fPIE -Xlinker -pie")

Bump required cmake version to 3.14 for CheckPIESupported.

References:
https://cmake.org/cmake/help/latest/prop_tgt/POSITION_INDEPENDENT_CODE.html
https://cmake.org/cmake/help/latest/module/CheckPIESupported.html#module:CheckPIESupported
2022-10-19 01:51:18 +08:00
Shengyun Zhou
2e77626d0f
wamrc: Support cross building and linking LLVM shared libs (#1578)
1. Support cross building wamrc and installing it
2. Remove PIE flag for Windows to fix compilation error when compiled by clang
3. Support linking LLVM shared libs to help build with system default or custom
   LLVM installation and reduce binary size.
2022-10-13 18:37:34 +08:00
YAMAMOTO Takashi
aba3707529
build_llvm.sh: enable to pass through arguments (#1506)
To make it simpler to specify `--project`.
2022-09-20 17:54:50 +08:00
Daniel Ludwig
046f5f2212
Fix Windows/MSVC build issues (#1498)
Fix two issues of building WAMR on Windows:
- The build_llvm.py script calls itself, spawning instances faster than they expire,
   which makes Python3 eat up the entire RAM in a pretty short time.
- The MSVC compiler doesn't support preprocessor statements inside macro expressions.
  Two places inside bh_assert() were found.
2022-09-17 21:16:38 +08:00
YAMAMOTO Takashi
1eedde7c33
Fix several issues related to AOT debug (#1492) 2022-09-16 12:06:46 +08:00
Wenyong Huang
22c235b5ec
Fix multi-module and some other issues (#1435)
Fix multi-module issue:
  don't call the sub module's function with "$sub_module_name$func_name"
Fix the aot_call_function free argv1 issue
Modify some API comments in wasm_export.h
Fix the wamrc help info
2022-08-31 17:38:38 +08:00
liang.he
717e8a48e2
Enable the semantic version mechanism for WAMR (#1374)
Use the semantic versioning (https://semver.org) to replace the current date
versioning system, which is more general and is requested by some developers,
e.g. issue #1357.

There are three parts in the new version string:
- major. Any incompatible modification on ABIs and APIs will lead to an increment
  in the value of major, which mainly includes: AOT calling conventions, AOT file
  format, wasm_export.h, wasm_c_api.h, and so on.
- minor. It represents new features, including MVP/POST-MVP features, libraries,
  WAMR private ones, and so one.
- patch. It represents patches.

The new version will start from 1.0.0. Update the help info and version showing for
iwasm and wamrc.
2022-08-18 19:01:05 +08:00
Wenyong Huang
9cf7b88bad
Enhance cmake makefiles (#1390)
Upgrade `cmake_minimum_required` from `(VERSION 2.8)` to `(VERSION 2.9)` to
yield the warning:
"Compatibility with CMake < 2.8.12 will be removed from a future version of CMake"

Add "-Wno-unused" for CMAKE_CXX_FLAGS to yield the compilation warnings
when build LLVM JIT.

Fix the link error when code coverage is enabled.
2022-08-18 16:27:01 +08:00
Xu Jun
9d689b3a22
Set noexecstack CXX link flags for wamrc (#1303)
Set noexecstack CXX link flags for wamrc to avoid generating binary
with exec stack.
2022-07-18 18:47:44 +08:00
Xu Jun
53b775aa4b
Support integrate 3rd-party toolchains into wamrc (#1237)
Support integrating 3rd-party toolchain llc compiler or asm compiler
into wamrc by setting environment variable WAMRC_LLC_COMPILER
or WAMRC_ASM_COMPILER, wamrc will use these tools to generate
object file from LLVM IR firstly, and then refactor the object file into
aot file.
2022-06-20 13:13:41 +08:00
Xu Jun
77595c9560
Support emit specified custom sections into AoT file (#1207)
And add API to get the content of custom section with
section name for both wasm file and aot file.
2022-06-10 21:51:13 +08:00
YAMAMOTO Takashi
a85f982297
wamr-compiler: Fix ubsan on macOS (#1105)
Supported on ubuntu as well
2022-04-20 16:59:13 +08:00
Wenyong Huang
21d89135da
Enhance uvwasi build process and fix wamrc windows build error (#1046)
And update related document
2022-03-15 21:43:55 +08:00
Wenyong Huang
25fc006c33
Refine call native function from AOT code (#1015)
When calling native function from AOT code, current implementation is to return
back to runtime to call aot_invoke_native, which calls wasm_runtime_invoke_native
and the latter calls assembly code. We did it before as there may be pointer and
string arguments to check and convert if the native function's registered signature
has character '*' and '$'.
As the built-in native function's signatures can be gotten in compilation time, we
check the pointer/string arguments and convert them into native address in AOT
code, and then invoke the native function directly, so as to improve performance.
2022-02-23 14:58:32 +08:00
lucianoiam
c8804c1ff9
Allow to build LLVM and wamrc on MinGW (#1013)
Use sysconfig.get_platform() to check whether the platform is MinGW,
and link necessary libraries to wamrc on MinGW.
2022-02-18 17:40:18 +08:00
Wenyong Huang
5631a2aa18
Use LLVM new pass manager for wamrc (#978)
Use LLVM new pass manager for wamrc to replace the legacy pass manger,
so as to gain better performance and reduce the compilation time.
Reference links:
- https://llvm.org/docs/NewPassManager.html
- https://blog.llvm.org/posts/2021-03-26-the-new-pass-manager

And add an option to use the legacy pm mode when building wamrc:
cmake .. -DWAMR_BUILD_LLVM_LEGACY_PM=1

For JIT mode, keep it unchanged as it only runs several function passes and
using new pass manager will increase the compilation time.

And refactor the codes of applying LLVM passes.
2022-01-24 11:10:37 +08:00
Wenyong Huang
4b5543cc1a
Enable Windows XIP (#944)
Enable running XIP file on Windows platform.
And add more strict checks for wamrc to report error when the input file
is same with output file, or the input file is AOT file but not wasm file.
2022-01-10 15:59:58 +08:00
Wenyong Huang
78308e7bda
GetCurrentThreadStackLimits dynamically for Windows platform (#939)
GetCurrentThreadStackLimits dynamically for Windows platform
according to suggestion in #902
And fix some compiling warnings on Windows platform
2022-01-07 17:00:38 +08:00
Wenyong Huang
cb51dbb513
Fix some issues on MacOS platform (#937)
Fix some issues on MacOS platform
- Enable libc-wasi by default
- Set target abi to "gnu" if it is not set for wamrc to avoid generating
  object file of unsupported Mach-O format
- Set `<vendor>-<sys>` info according to target abi for wamrc to support
  generating AOT file for other OSs but not current host
- Set cpu name if arch/abi/cpu are not set to avoid checking SIMD
  capability failed
- Set size level to 1 for MacOS/Windows platform to avoid relocation type
  unsupported warning
- Clear posix_memmap.c compiling warning
- Fix spec case test script issues, enable test spec cases on MacOS

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
2022-01-07 09:53:48 +08:00
liang.he
1dccf39d16
Enable ref types and bulk memory by default for wamrc (#838)
Enable ref types feature and bulk memory feature by default for wamrc
and provide "--disable-ref-types", "--disable-bulk-memory"  to disable
them.
And remove the ref_type_flag option in wasm_loader.c which is used to
control whether to enable ref types or not when ENABLE_REF_TYPES
macro is enabled in wamrc. As the wasm binary format with ref types
is compatible with the binary format before, we can remove the option.
Also update the spec test scripts.
2021-11-18 17:42:49 +08:00
Wenyong Huang
64be6ec9a7
Fix potential pointer overflows (#826)
Fix some potential pointer overflows in aot applying relocations and
several other places.
And add sanitizer compiler flags to wamrc CMakeLists.txt to detect
such issues.
2021-11-15 10:57:37 +08:00
Wenyong Huang
a1ad950ae1
Enable LLVM link time optimization (LTO) for AOT (#831)
Enable LLVM link time optimization for AOT and enable it by default,
and provide "wamrc --disable-llvm-lto" option to disable it.
And modify the spec test script, disable it by default when testing
spec cases.

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
2021-11-13 16:59:35 +08:00
YAMAMOTO Takashi
d1be75c4a1
wamrc: return a non-zero exit code on failure (#822) 2021-11-09 13:13:20 +08:00
Wenyong Huang
32242988ed
Apply clang-format for more source files (#795)
Apply clang-format for C source files in folder core/app-mgr,
core/app-framework, and test-tools.
And rename folder component_test to component-test, update
zephyr build document.

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
2021-10-21 13:58:34 +08:00
Wenyong Huang
03494f9487
Refactor CI rules: merge ubuntu/macos/android, enable spec test (#777)
Refactor CI rules:
- merge ubuntu/macos/android into one job file
- add job file to test spec cases
- add compilation for lazy jit, debug interpreter and debug aot
- add compilation for performance profile, dump call stack and mini-loader
- re-org llvm build script to build lldb

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
2021-10-11 20:32:29 +08:00
Wenyong Huang
9ef37dd781
Implement source debugging for interpreter and AOT (#769)
Implement source debugging feature for classic interpreter and AOT:
- use `cmake -DWAMR_BUILD_DEBUG_INTERP=1` to enable interpreter debugging
- use `cmake -DWAMR_BUILD_DEBUG_AOT=1` to enable AOT debugging

See doc/source_debugging.md for more details.
2021-09-29 13:36:46 +08:00
Wenyong Huang
7be0d385a6
Implement SIMD latest opcodes and update LLVM to 13.0 (#758)
Implement the latest SIMD opcodes and update LLVM 13.0,
update the llvm build scripts, update the sample workloads‘ build scripts,
and build customized wasi-sdk to build some workloads.
Also refine the CI rules.

Signed-off-by: Wenyong Huang <wenyong.huang@intel.com>
2021-09-17 19:12:57 +08:00