29 Commits

Author	SHA1	Message	Date
liang.he	0599351262	wasi-nn: Add a new target for llama.cpp as a wasi-nn backend (#3709) ... Minimum support: - [x] accept (WasmEdge) customized model parameters. metadata. - [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [x] basic - [x] chatml - [x] gemma - [x] llama - [x] qwen --- In the future, to support if required: - [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [ ] command-r. (>70G memory requirement) - [ ] embedding. (embedding mode) - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output) - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`) - [ ] llava. (image representation) - [ ] llava-base64-stream. (image representation) - [ ] multimodel. (image representation) - [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)	2024-09-10 08:45:18 +08:00
liang.he	140ff25d46	wasi-nn: Apply new architecture (#3692) ... ps. https://github.com/bytecodealliance/wasm-micro-runtime/issues/3677	2024-08-13 09:14:52 +08:00
dependabot[bot]	0a56abc6d6	build(deps): bump tensorflow in /core/iwasm/libraries/wasi-nn/test (#3675) ... Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.11.1 to 2.12.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.11.1...v2.12.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-02 09:17:12 +08:00
liang.he	058bc47102	[wasi-nn] Add a new wasi-nn backend openvino (#3603)	2024-07-22 17:16:41 +08:00
tonibofarull	77da87ca51	wasi-nn: Use numpy v1 in wasi-nn test requirements.txt (#3582) ... We need to fix numpy version since the latest is incompatible. > A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.0 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.	2024-07-02 09:39:46 +08:00
liang.he	d36160b294	wasi-nn: Add wasmedge-wasinn-example as smoke test (#3554)	2024-06-24 12:03:08 +08:00
liang.he	db025e457a	sync up with latest wasi-nn spec (#3530)	2024-06-17 14:58:09 +08:00
liang.he	f844b33b2d	Make wasi-nn backends as separated shared libraries (#3509) ... - All files under *core/iwasm/libraries/wasi-nn* are compiled as shared libraries - *wasi-nn.c* is shared between backends - Every backend has a separated shared library - If wasi-nn feature is enabled, iwasm will depend on shared library libiwasm.so instead of linking static library libvmlib.a	2024-06-14 12:06:56 +08:00
liang.he	028f43bc18	Fix compilation warnings of wasi-nn (#3497)	2024-06-07 10:49:44 +08:00
liang.he	bd44117676	Add wasi-nn example as smoke test case (#3501)	2024-06-07 10:26:09 +08:00
Xu Jinyang	cef88deedb	Add wasi_ephemeral_nn module support (#3241) ... Add `wasi_ephemeral_nn` module support with optional cmake variable, which was mentioned in #3229.	2024-03-21 21:05:34 +08:00
Wenyong Huang	0ee5ffce85	Refactor APIs and data structures as preliminary work for Memory64 (#3209) ... # Change the data type representing linear memory address from u32 to u64 ## APIs signature changes - (Export)wasm_runtime_module_malloc - wasm_module_malloc - wasm_module_malloc_internal - aot_module_malloc - aot_module_malloc_internal - wasm_runtime_module_realloc - wasm_module_realloc - wasm_module_realloc_internal - aot_module_realloc - aot_module_realloc_internal - (Export)wasm_runtime_module_free - wasm_module_free - wasm_module_free_internal - aot_module_malloc - aot_module_free_internal - (Export)wasm_runtime_module_dup_data - wasm_module_dup_data - aot_module_dup_data - (Export)wasm_runtime_validate_app_addr - (Export)wasm_runtime_validate_app_str_addr - (Export)wasm_runtime_validate_native_addr - (Export)wasm_runtime_addr_app_to_native - (Export)wasm_runtime_addr_native_to_app - (Export)wasm_runtime_get_app_addr_range - aot_set_aux_stack - aot_get_aux_stack - wasm_set_aux_stack - wasm_get_aux_stack - aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert and jit_check_app_addr_and_convert - wasm_exec_env_set_aux_stack - wasm_exec_env_get_aux_stack - wasm_cluster_create_thread - wasm_cluster_allocate_aux_stack - wasm_cluster_free_aux_stack ## Data structure changes - WASMModule and AOTModule - field aux_data_end, aux_heap_base and aux_stack_bottom - WASMExecEnv - field aux_stack_boundary and aux_stack_bottom - AOTCompData - field aux_data_end, aux_heap_base and aux_stack_bottom - WASMMemoryInstance(AOTMemoryInstance) - field memory_data_size and change __padding to is_memory64 - WASMModuleInstMemConsumption - field total_size and memories_size - WASMDebugExecutionMemory - field start_offset and current_pos - WASMCluster - field stack_tops ## Components that are affected by the APIs and data structure changes - libc-builtin - libc-emcc - libc-uvwasi - libc-wasi - Python and Go Language Embedding - Interpreter Debug engine - Multi-thread: lib-pthread, wasi-threads and thread manager	2024-03-12 11:38:50 +08:00
Zhen Kong	1a88104160	Remove module instance from hashmap in wasi_nn_destroy (#2613) ... When destroying wasi-nn context, module instance should be also removed from hashmap to avoid memory leak.	2023-10-03 08:33:11 +08:00
tonibofarull	b45d014112	wasi-nn: Improve TPU support (#2447) ... 1. Allow TPU and GPU support at the same time. 2. Add Dockerfile to run example with [Coral USB](https://coral.ai/products/accelerator/).	2023-08-14 20:03:56 +08:00
tonibofarull	0b0af1b3df	wasi-nn: Support uint8 quantized networks (#2433) ... Support (non-full) uint8 quantized networks. Inputs and outputs are still required to be `float`. The (de)quantization is done internally by wasi-nn. Example generated from `quantized_model.py`:  Visualization with [netron](https://netron.app/).	2023-08-11 07:55:40 +08:00
tonibofarull	ab96e01f5e	wasi-nn: Add support of wasi-nn as shared lib (#2310) ... ## Context Currently, WAMR supports compiling iwasm with flag `WAMR_BUILD_WASI_NN`. However, there are scenarios where the user might prefer having it as a shared library. ## Proposed Changes Decouple wasi-nn context management by internally managing the context given a module instance reference.	2023-06-27 18:18:26 +08:00
tonibofarull	f5c5a83331	wasi-nn: Improve tests paths for local dev (#2309) ... ## Context Path to models use `/assets` for testing inside docker. While testing directly from the repo we are forced to use soft-links or modify the paths. ## Proposed Changes Use relative path and adjust docker volumes in docs.	2023-06-27 08:07:30 +08:00
tonibofarull	ac9e789951	wasi-nn: Simplify cmake and headers' location (#2308) ... Major changes: - Public headers inside `wasi-nn/include` - Put cmake files in `cmake` folder - Make linux iwasm link with `${WASI_NN_LIBS}` so iwasm can enable wasi-nn	2023-06-26 09:29:05 +08:00
TianlongLiang	8f82b1e8a2	Fix dockerfile linter warnings (#2291)	2023-06-15 16:52:48 +08:00
YAMAMOTO Takashi	6e3c3fe9ec	Fix build error with LLVM 16 (#2259)	2023-06-06 13:45:18 +08:00
ayakoakasaka	89be5622a5	wasi-nn: Add external delegation to support several NPU/GPU (#2162) ... Add VX delegation as an external delegation of TFLite, so that several NPU/GPU (from VeriSilicon, NXP, Amlogic) can be controlled via WASI-NN. Test Code can work with the X86 simulator.	2023-05-05 16:29:36 +08:00
ayakoakasaka	ed6b8efade	Avoid re-installing if Tensorflow is already installed for WASI-NN (#2148) ... Since the Tensorflow library is already installed in many cases(especially in the case of the embedded system), move the installation code to find_package.	2023-04-27 08:19:18 +08:00
dependabot[bot]	0ce39a1368	Bump tensorflow in /core/iwasm/libraries/wasi-nn/test (#2061) ... Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.10.1 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.10.1...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2023-03-28 16:36:59 +08:00
tonibofarull	a15a731e12	wasi-nn: Support multiple TFLite models (#2002) ... Remove restrictions: - Only 1 WASM app at a time - Only 1 model at a time - `graph` and `graph-execution-context` are ignored Refer to previous document: e8d718096d/core/iwasm/libraries/wasi-nn/README.md	2023-03-08 15:54:06 +08:00
Wenyong Huang	9b9ae0cfac	Update cmake files and wamr-test-suites to support collect code coverage (#1992) ... Support collecting code coverage with wamr-test-suites script by using lcov and genhtml tools, eg.: cd tests/wamr-test-suites ./test_wamr.sh -s spec -b -P -C The default code coverage and html files are generated at: tests/wamr-test-suites/workspace/wamr.lcov tests/wamr-test-suites/workspace/wamr-lcov.zip And update wamr-test-suites scripts to support testing GC spec cases to avoid frequent synchronization conflicts between branch main and dev/gc.	2023-02-28 17:38:18 +08:00
tonibofarull	1614ce12fa	wasi-nn: Enable GPU support (#1922) ... - Split logic in several dockers - runtime: wasi-nn-cpu and wasi-nn- Nvidia-gpu. - compilation: wasi-nn-compile. Prepare the testing wasm and generates the TFLites. - Implement GPU support for TFLite with Opencl.	2023-02-02 08:09:46 +08:00
tonibofarull	9eed6686df	Refactor WASI-NN to simplify the support for multiple frameworks (#1834) ... - Reorganize the library structure - Use the latest version of `wasi-nn` wit (Oct 25, 2022): 0f77c48ec1/wasi-nn.wit.md - Split logic that converts WASM structs to native structs in a separate file - Simplify addition of new frameworks	2023-01-25 18:32:40 +08:00
dependabot[bot]	2c5e7bc650	Bump tensorflow from 2.10.0 to 2.10.1 in /core/iwasm/libraries/wasi-nn/test (#1735) ... Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.10.0 to 2.10.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.10.0...v2.10.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2022-11-22 16:21:06 +08:00
tonibofarull	e53ab91439	Integrate WASI-NN into WAMR (#1521) ... Initial integration of WASI-NN based on #1225: - Implement the library core/iwasm/libraries/wasi-nn - Support TensorFlow, CPU, F32 at the first stage - Add cmake variable `-DWAMR_BUILD_WASI_NN` - Add test case based on Docker image and update document Refer to #1573	2022-10-12 12:09:29 +08:00