397 Commits

Author	SHA1	Message	Date
YAMAMOTO Takashi	2372a472aa	wasi-nn: make the host use the wasi_ephemeral_nn version of tensor_data (#4411) ... the motivations: * make the actual input size available to the backends. (currently the backends have to make a guess from shape/type.) * make the host logic look a bit similar to wasi_ephemeral_nn. this is a backend api/abi change.	2025-06-27 07:41:42 +08:00
YAMAMOTO Takashi	a7aae9d2cc	wasi_nn_llamacpp.c: make this compilable (#4403)	2025-06-26 07:05:45 +08:00
YAMAMOTO Takashi	8289452abb	wasi_nn_tensorflowlite.cpp: fix get_output return size (#4390) ... it should be byte size, not the number of (fp32) values. i'm ambivalent about how to deal with the compatibility for the legacy wamr-specific "wasi_nn". for now, i avoided changing it. (so that existing tests using the legacy abi, namely test_tensorflow.c and test_tensorflow_quantized.c, passes as they are.) if we have any users who still want to use the legacy abi, i suppose they consider the compatibility is more important than the consistency with other backends. cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4376	2025-06-24 20:38:19 +08:00
YAMAMOTO Takashi	70c39bae77	wasi-nn: fix context lifetime issues (#4396) ... * wasi-nn: fix context lifetime issues use the module instance context api instead of trying to roll our own with a hashmap. this fixes context lifetime problems mentioned in https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313. namely, * wasi-nn resources will be freed earlier now. before this change, they used to be kept until the runtime shutdown. (wasm_runtime_destroy) after this change, they will be freed together with the associated instances. * wasm_module_inst_t pointer uniqueness assumption (which is wrong after wasm_runtime_deinstantiate) was lifted. as a side effect, this change also makes a context shared among threads within a cluster. note that this is a user-visible api/abi breaking change. before this change, wasi-nn "handles" like wasi_ephemeral_nn_graph were thread-local. after this change, they are shared among threads within a cluster, similarly to wasi file descriptors. spec-wise, either behavior should be ok simply because wasi officially doesn't have threads yet. althogh i feel the latter semantics is more intuitive, if your application depends on the thread-local behavior, this change breaks your application. tested with wamr-wasi-extensions/samples/nn-cli, modified to call each wasi-nn operations on different threads. (if you are interested, you can find the modification at https://github.com/yamt/wasm-micro-runtime/tree/yamt-nn-wip-20250619.) cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313 https://github.com/bytecodealliance/wasm-micro-runtime/issues/2430 * runtime_lib.cmake: enable WAMR_BUILD_MODULE_INST_CONTEXT for wasi-nn as we do for wasi (WAMR_BUILD_LIBC_WASI)	2025-06-24 20:37:56 +08:00
YAMAMOTO Takashi	f449b79a31	wasi_nn_openvino.c: implement multiple models per instance (#4380) ... tested with two models: ``` --load-graph=id=graph1,file=public/license-plate-recognition-barrier-0007/FP32/license-plate-recognition-barrier-0007.xml,file=public/license-plate-recognition-barrier-0007/FP32/license-plate-recognition-barrier-0007.bin \ --load-graph=id=graph2,file=classify/model.xml,file=classify/model.bin \ --init-execution-context=id=exec1,graph-id=graph1 \ --init-execution-context=id=exec2,graph-id=graph2 \ --set-input=context-id=exec1,dim=1,dim=24,dim=94,dim=3,file=out.bin \ --set-input=context-id=exec2,file=classify/banana-3x224x224-bgr.bin,dim=1,dim=3,dim=224,dim=224 \ --compute=context-id=exec1 \ --compute=context-id=exec2 \ --get-output=context-id=exec1,file=exec1-result.bin \ --get-output=context-id=exec2,file=exec2-result.bin ``` a detailed HOWTO: https://github.com/bytecodealliance/wasm-micro-runtime/pull/4380#issuecomment-2986882718	2025-06-20 15:50:29 +08:00
YAMAMOTO Takashi	ea408ab6c0	wasi-nn: add minimum serialization on WASINNContext (#4387) ... currently this is not necessary because context (WASINNContext) is local to instance. (wasm_module_instance_t) i plan to make a context shared among instances in a cluster when fixing https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313. this is a preparation for that direction. an obvious alternative is to tweak the module instance context APIs to allow declaring some kind of contexts instance-local. but i feel, in this particular case, it's more natural to make "wasi-nn handles" shared among threads within a "process". note that, spec-wise, how wasi-nn behaves wrt threads is not defined at all because wasi officially doesn't have threads yet. i suppose, at this point, that how wasi-nn interacts with wasi-threads is something we need to define by ourselves, especially when we are using an outdated wasi-nn version. with this change, if a thread attempts to access a context while another thread is using it, we simply make the operation fail with the "busy" error. this is intended for the mimimum serialization to avoid problems like crashes/leaks/etc. this is not intended to allow parallelism or such. no functional changes are intended at this point yet. cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313 https://github.com/bytecodealliance/wasm-micro-runtime/issues/2430	2025-06-20 09:48:55 +08:00
YAMAMOTO Takashi	71c07f3e4e	deprecate legacy WAMR-specific "wasi_nn" module (#4382) ... wasi_nn.h: deprecate legacy "wasi_nn" cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4326	2025-06-19 14:32:26 +08:00
YAMAMOTO Takashi	aa53d648fa	wasi-nn: fix tensor_data abi for wasi_ephemeral_nn (#4379) ... it's "(list u8)" in the witx definition. the new definition matches both of our own host definition (struct tensor_wasm) and wasmtime. cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4352	2025-06-19 14:18:36 +08:00
YAMAMOTO Takashi	a29f3943ef	core/iwasm/libraries/wasi-nn/test: use the correct version of keras (#4383)	2025-06-18 19:24:06 +08:00
YAMAMOTO Takashi	db7714f0f5	wasi_nn_tensorflowlite.cpp: reject non-fp32 input earlier (#4388) ... this backend assumes fp32 here and there. it's safer to reject unexpected inputs explicitly.	2025-06-18 19:08:57 +08:00
YAMAMOTO Takashi	4bf799c3af	core/iwasm/libraries/wasi-nn/test/build.sh: add a tip for intel mac (#4389) ... i keep forgetting this and had to re-investigate it at least twice. hopefully this can be helpful for others too.	2025-06-18 19:06:57 +08:00
YAMAMOTO Takashi	cba9001749	wasi-nn: don't try to deinit uninitialized backend (#4375) ... cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4339	2025-06-17 17:40:53 +08:00
YAMAMOTO Takashi	c9b8c16088	wasi_nn_openvino.c: remove pre/postprocessing and layout assumptions (#4361) ... as wasi-nn doesn't have these concepts, the best we can do without risking breaking certain applications here is to pass through tensors as they are. this matches wasmtime's behavior. tested with: * wasmtime classification-example (with this change, this example fails on tensor size mismatch instead of implicitly resizing it.) * license-plate-recognition-barrier-0007, a converted version with non-fp32 output. [1] (with this change, this model outputs integers as expected.) [1] cd7ebe313b/models/public/license-plate-recognition-barrier-0007	2025-06-17 13:01:46 +08:00
Su Yihan	6dfb410869	send an empty/error reply from server (#4362) ... Signed-off-by: Su Yihan <yihan.su@intel.com>	2025-06-17 13:00:58 +08:00
YAMAMOTO Takashi	2f0750a6fe	wasi_nn_openvino.c: add a missing buffer overflow check in get_output (#4353) ... cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4351	2025-06-17 11:17:00 +08:00
YAMAMOTO Takashi	20be1d33fe	wasi_ephemeral_nn.h: prefix identfiers to avoid too generic names (#4358)	2025-06-17 11:15:01 +08:00
YAMAMOTO Takashi	745da82cd6	wasi_nn_openvino.c: remove broken xml check (#4365) ... `xml.buf[xml.size]` check is broken because it accesses past the end of the buffer. anyway, openvino doesn't seem to care the NUL termination.	2025-06-17 11:02:36 +08:00
YAMAMOTO Takashi	0d001c4c38	wasi-nn: fix backend leak on multiple loads (#4366) ... cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4340	2025-06-17 11:01:07 +08:00
TianlongLiang	8e60feb181	Collective fix for typos and minor bugs (#4369)	2025-06-17 09:26:00 +08:00
YAMAMOTO Takashi	5478d267f4	wasi_nn_openvino.c: remove the tensor layout adjustment logic (#4308) ... the logic in question seems like an attempt to work around some application bugs. my wild guess is that it was for classification-example. cf. https://github.com/bytecodealliance/wasmtime/issues/10867	2025-06-12 09:34:14 +08:00
YAMAMOTO Takashi	3a087c4244	wamr-wasi-extensions: add a cmake package to provide our wasi extension (#4344) ... * wasi_ephemeral_nn.h: add a convenience wrapper header * wamr-wasi-extensions: add a cmake package to provide our wasi extension the sample app was tested with: * wasmtime * iwasm with https://github.com/bytecodealliance/wasm-micro-runtime/pull/4308 currently only contains wasi-nn. maybe it makes sense to add lib-socket things as well. cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4288	2025-06-12 09:33:25 +08:00
YAMAMOTO Takashi	c932597057	wasi_nn_types.h: remove a seemingly stale comment (#4348)	2025-06-12 09:29:59 +08:00
YAMAMOTO Takashi	7f968f5926	wasi_socket_ext.c: avoid tls to make this library-friendly (#4338)	2025-06-11 07:46:35 +08:00
YAMAMOTO Takashi	ea5757f1d7	wasi-nn: do not assign wasi_nn_ctx->backend multiple times (#4329)	2025-06-09 11:36:31 +08:00
YAMAMOTO Takashi	4d6b8dcd5d	wasi_nn.h: make this compatible with wasi_ephemeral_nn (#4330) ... - wasi_nn.h: make this compatible with wasi_ephemeral_nn cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4323 - fix WASM_ENABLE_WASI_EPHEMERAL_NN build this structure is used by host logic as well. ideally definitions for wasm and host should be separated. until it happens, check __wasm__ to avoid the breakage.	2025-06-09 11:36:05 +08:00
YAMAMOTO Takashi	933f8124b0	wasi-nn: fix the size of tensor->type (#4333) ... * this enum is (@witx tag u8) in witx * it seems that some wasm modules actually use non-zero padding and cause errors * it's a bad practice to use C enum for ABI description anyway	2025-06-06 15:08:18 +08:00
YAMAMOTO Takashi	769d16eaab	wasi-nn: move some host-only things out of wasi_nn_types.h (#4334) ... cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4324	2025-06-06 15:07:29 +08:00
YAMAMOTO Takashi	79cb4366ae	wasi-nn: remove unused wasi_nn_dump_tensor_dimension prototype (#4325)	2025-06-05 09:48:28 +08:00
YAMAMOTO Takashi	b20ebc2724	wasi_nn.h: add import_name attribute (#4328) ... this would fix undefined symbol errors by making it clear these functions are imported. references: e2c698c7e8/llvm/lib/MC/WasmObjectWriter.cpp (L1798-L1799) e2c698c7e8/llvm/lib/Object/WasmObjectFile.cpp (L749-L752) e2c698c7e8/lld/wasm/Symbols.cpp (L203) e2c698c7e8/lld/wasm/Relocations.cpp (L36-L40)	2025-06-05 09:48:00 +08:00
YAMAMOTO Takashi	85efe08431	wasi-nn: protect the backend lookup table with a lock (#4319) ... this would avoid potential issues when multiple instances happen to make an attempt to load a backend at the same time. Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/4314	2025-06-05 09:19:46 +08:00
YAMAMOTO Takashi	6a00874f2f	wasi_nn_openvino.c: make this buildable (#4305)	2025-06-03 13:28:13 +08:00
YAMAMOTO Takashi	61cb97221e	wasi-nn: fix shared library filenames for macOS (#4306) ... tested with openvino	2025-06-03 13:23:19 +08:00
YAMAMOTO Takashi	ae6e490ad5	fix wasi-nn abi definitions (#4307) ... sync with a more appropriate version of the definitions. as we use the "wasi_ephemeral_nn", which is p1-based, it seems more appropriate to use definitions from witx, not wit. it's a bit unfortunate p2-based wasi-nn made gratuitous changes like this from p1. note: this is an ABI change.	2025-06-03 13:22:48 +08:00
YAMAMOTO Takashi	16c46751ac	wasi-nn: remove "backends" argument from detect_and_load_backend() (#4309) ... it seems meaningless and quite confusing to access a table with two aliases ("lookup" and "backends") within a function. no functional changes are intended.	2025-06-03 13:22:27 +08:00
YAMAMOTO Takashi	1c12a32066	wasi_nn_openvino.c: fix a few printf formats (#4310)	2025-06-03 13:21:32 +08:00
liang.he	938503af38	Bump uvwasi to latest commit #392e1f1 (#4312)	2025-06-03 13:21:17 +08:00
hongxia	aa1ff778b9	add load_by_name in wasi-nn (#4298)	2025-06-03 06:26:58 +08:00
hongxia	3ab9f84026	Dockerfile.vx-delegate build error fix (#4273) ... - specify tensorflow version & bugfix	2025-05-28 20:29:41 +08:00
liang.he	28702edaf7	Merge commit from fork	2025-05-14 12:43:55 +08:00
liang.he	8f3961026e	fix: improve error handling of snprintf() in send_thread_stop_status() (#4234) ... Prevent `MAX_PACKET_SIZE - len` from overflowing.	2025-05-09 17:14:20 +08:00
liang.he	1a72dcf34f	Improve readlinkat_dup() to handle symlink size correctly (#4229) ... * In readlinkat_dup(), use fstatat() to estimate size first. * Reduce additional space in samples/file	2025-05-09 14:01:29 +08:00
TianlongLiang	ff2768775c	fix potential memory leak (#4205)	2025-04-22 12:08:25 +08:00
YAMAMOTO Takashi	0ba6532636	debug-engine: fix a few type mismatches (#4189) ... - use strict prototypes complained by GCC `-Wstrict-prototypes` - use `int*` instead of `int32*` Note: on some targets, int32_t is a long. for example, GCC shipped with the recent ESP-IDF has such a configuration. - https://github.com/apache/nuttx/issues/15755#issuecomment-2635652808 - https://github.com/apache/nuttx/pull/16022 - https://docs.espressif.com/projects/esp-idf/en/stable/esp32/migration-guides/release-5.x/5.0/gcc.html#espressif-toolchain-changes	2025-04-17 00:07:08 +08:00
liang.he	d085d1ccf7	Keep fix the CMake compatibility issue (#4180) ... ``` CMake Error at CMakeLists.txt:4 (cmake_minimum_required): Compatibility with CMake < 3.5 has been removed from CMake. Update the VERSION argument <min> value. Or, use the <min>...<max> syntax to tell CMake that the project requires at least <min> but has been updated to work with policies introduced by <max> or earlier. Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway. ```	2025-04-15 12:51:19 +08:00
liang.he	2a2632444b	Refactor Dockerfile and update .dockerignore for wasi-nn tests; adjust map-dir parameters in smoke test script (#4158)	2025-04-10 11:59:59 +08:00
liang.he	159b69da38	Update cmake min to 3.14 (#4175) ... 3.14 is used and tested by linux mini-product to fix ``` CMake Error at CMakeLists.txt:4 (cmake_minimum_required): Compatibility with CMake < 3.5 has been removed from CMake. Update the VERSION argument <min> value. Or, use the <min>...<max> syntax to tell CMake that the project requires at least <min> but has been updated to work with policies introduced by <max> or earlier. Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway. ```	2025-04-02 07:13:53 +08:00
Marcin Kolny	efa8019bdb	Merge dev/simd for fast-interp (#4131) ... * Implement the first few SIMD opcodes for fast interpreter (v128.const, v128.any_true) (#3818) Tested on the following code: ``` (module (import "wasi_snapshot_preview1" "proc_exit" (func $proc_exit (param i32))) (memory (export "memory") 1) ;; WASI entry point (func $main (export "_start") v128.const i8x16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 v128.any_true if unreachable end v128.const i8x16 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 v128.any_true i32.const 0 i32.eq if unreachable end i32.const 0 call $proc_exit ) ) ``` * implement POP_V128() This is to simplify the simd implementation for fast interpreter * Add all SIMD operations into wasm_interp_fast switch * Add V128 comparison operations Tested using ``` (module (import "wasi_snapshot_preview1" "proc_exit" (func $proc_exit (param i32))) (memory (export "memory") 1) (func $assert_true (param v128) local.get 0 v128.any_true i32.eqz if unreachable end ) (func $main (export "_start") ;; Test v128.not v128.const i8x16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 v128.not v128.const i8x16 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 i8x16.eq call $assert_true ;; Test v128.and v128.const i8x16 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 v128.const i8x16 255 255 0 0 255 255 0 0 255 255 0 0 255 255 0 0 v128.and v128.const i8x16 255 255 0 0 0 0 0 0 255 255 0 0 0 0 0 0 i8x16.eq call $assert_true ;; Test v128.andnot v128.const i8x16 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 v128.const i8x16 255 255 0 0 255 255 0 0 255 255 0 0 255 255 0 0 v128.andnot v128.const i8x16 0 0 255 255 0 0 0 0 0 0 255 255 0 0 0 0 i8x16.eq call $assert_true ;; Test v128.or v128.const i8x16 255 255 0 0 0 0 255 255 255 255 0 0 0 0 255 0 v128.const i8x16 0 0 255 255 255 255 0 0 0 0 255 255 255 255 0 0 v128.or v128.const i8x16 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 i8x16.eq call $assert_true ;; Test v128.xor v128.const i8x16 255 255 0 0 255 255 0 0 255 255 0 0 255 255 0 0 v128.const i8x16 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 v128.xor v128.const i8x16 0 0 255 255 255 255 0 0 0 0 255 255 255 255 0 0 i8x16.eq call $assert_true i32.const 0 call $proc_exit ) ) ``` * Add first NEON SIMD opcode implementations to fast interpreter (#3859) Add some implementations of SIMD opcodes using NEON instructions. Tested using: ```wast (module (import "wasi_snapshot_preview1" "proc_exit" (func $proc_exit (param i32))) (memory (export "memory") 1) (func $assert_true (param v128) local.get 0 v128.any_true i32.eqz if unreachable end ) (func $main (export "_start") i32.const 0 i32.const 32 memory.grow drop i32.const 0 v128.const i8x16 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 v128.store i32.const 0 v128.load v128.const i8x16 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 i8x16.eq call $assert_true i32.const 16 v128.const i8x16 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 v128.store i32.const 16 v128.load v128.const i8x16 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 i8x16.eq call $assert_true i32.const 0 v128.load v128.const i8x16 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 i8x16.eq call $assert_true drop i32.const 0 i32.const 1 memory.grow drop i32.const 0 i64.const 0x7F80FF017E02FE80 i64.store i32.const 0 v128.load8x8_s v128.const i16x8 127 -128 -1 1 126 2 -2 -128 i16x8.eq call $assert_true i32.const 0 i64.const 0x80FE027E01FF807F i64.store i32.const 0 v128.load8x8_u v128.const i16x8 128 254 2 126 1 255 128 127 i16x8.eq call $assert_true i32.const 0 i64.const 0x8000FFFE7FFF0001 i64.store i32.const 0 v128.load16x4_s v128.const i32x4 -32768 -2 32767 1 i32x4.eq call $assert_true i32.const 0 i64.const 0x8000FFFE7FFF0001 i64.store i32.const 0 v128.load16x4_u v128.const i32x4 32768 65534 32767 1 i32x4.eq call $assert_true i32.const 0 i64.const 0x8000000000000001 i64.store i32.const 0 v128.load32x2_s v128.const i64x2 -2147483648 1 i64x2.eq call $assert_true i32.const 0 i64.const 0x8000000000000001 i64.store i32.const 0 v128.load32x2_u v128.const i64x2 2147483648 1 i64x2.eq call $assert_true call $proc_exit ) ) ``` * Emit imm for lane extract and replace (#3906) * Fix replacement value not being correct (#3919) * Implement load lanes opcodes for wasm (#3942) * Implement final SIMD opcodes: store lane (#4001) * Fix load/store (#4054) * Correctly use unsigned functions (#4055) * implement local and function calls for v128 in the fast interpreter * Fix splat opcodes, add V128 handling in preserve_referenced_local and reserve_block_ret * Fix incorrect memory overflow values + SIMD ifdefs * Fix load/load_splat macros * correct endif wasm loader * Update core/iwasm/interpreter/wasm_opcode.h * Fix spec tests when WASM_CPU_SUPPORTS_UNALIGNED_ADDR_ACCESS is 0 * Resolve merge conflicts arising from main -> dev/simd_for_interp and implement fast interpreter const offset loader support for V128 * Enable SIMDe tests on CI * Document WAMR_BUILD_LIB_SIMDE --------- Co-authored-by: James Marsh <mrshnja@amazon.co.uk> Co-authored-by: jammar1 <108334558+jammar1@users.noreply.github.com> Co-authored-by: Maks Litskevich <makslit@amazon.com> Co-authored-by: Marcin Kolny <marcin.kolny@gmail.com> Co-authored-by: Wenyong Huang <wenyong.huang@intel.com>	2025-03-20 14:23:20 +08:00
Zhen Kong	dde6477fa5	Fix iwasm build error when WAMR_BUILD_WASI_NN enabled ... A recent change on ./product-mini/platforms/linux/CMakeLists.txt renamed libiwasm to vmlib, but wasi-nn.cmake still wants to link libiwasm.so. Replace libiwasm with vmlib in wasi-nn.cmake to resolve iwasm build error when WAMR_BUILD_WASI_NN enabled.	2025-03-13 17:08:22 +00:00
Huang Qi	412631ac13	fix: correct typos and improve comments across multiple files by codespell (#4116) ... Signed-off-by: Huang Qi <huangqi3@xiaomi.com>	2025-03-07 08:21:54 +08:00
Marcin Kolny	b2c7cb2375	Use wasm32-wasip1 instead of wasm32-wasi target for rust code (#4057) ... Rust compiler previously deprecated, and now removed the wasm32-wasi target and replaced it with wasm32-wasip1. This change updates all the occurrences of wasm32-wasi in the context of Rust compilation. covers the wasi-nn/test.	2025-02-05 11:31:49 +08:00