16 Commits

Author	SHA1	Message	Date
liang.he	0599351262	wasi-nn: Add a new target for llama.cpp as a wasi-nn backend (#3709) ... Minimum support: - [x] accept (WasmEdge) customized model parameters. metadata. - [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [x] basic - [x] chatml - [x] gemma - [x] llama - [x] qwen --- In the future, to support if required: - [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [ ] command-r. (>70G memory requirement) - [ ] embedding. (embedding mode) - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output) - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`) - [ ] llava. (image representation) - [ ] llava-base64-stream. (image representation) - [ ] multimodel. (image representation) - [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)	2024-09-10 08:45:18 +08:00
liang.he	140ff25d46	wasi-nn: Apply new architecture (#3692) ... ps. https://github.com/bytecodealliance/wasm-micro-runtime/issues/3677	2024-08-13 09:14:52 +08:00
liang.he	058bc47102	[wasi-nn] Add a new wasi-nn backend openvino (#3603)	2024-07-22 17:16:41 +08:00
liang.he	db025e457a	sync up with latest wasi-nn spec (#3530)	2024-06-17 14:58:09 +08:00
liang.he	f844b33b2d	Make wasi-nn backends as separated shared libraries (#3509) ... - All files under *core/iwasm/libraries/wasi-nn* are compiled as shared libraries - *wasi-nn.c* is shared between backends - Every backend has a separated shared library - If wasi-nn feature is enabled, iwasm will depend on shared library libiwasm.so instead of linking static library libvmlib.a	2024-06-14 12:06:56 +08:00
liang.he	028f43bc18	Fix compilation warnings of wasi-nn (#3497)	2024-06-07 10:49:44 +08:00
Xu Jinyang	cef88deedb	Add wasi_ephemeral_nn module support (#3241) ... Add `wasi_ephemeral_nn` module support with optional cmake variable, which was mentioned in #3229.	2024-03-21 21:05:34 +08:00
Wenyong Huang	0ee5ffce85	Refactor APIs and data structures as preliminary work for Memory64 (#3209) ... # Change the data type representing linear memory address from u32 to u64 ## APIs signature changes - (Export)wasm_runtime_module_malloc - wasm_module_malloc - wasm_module_malloc_internal - aot_module_malloc - aot_module_malloc_internal - wasm_runtime_module_realloc - wasm_module_realloc - wasm_module_realloc_internal - aot_module_realloc - aot_module_realloc_internal - (Export)wasm_runtime_module_free - wasm_module_free - wasm_module_free_internal - aot_module_malloc - aot_module_free_internal - (Export)wasm_runtime_module_dup_data - wasm_module_dup_data - aot_module_dup_data - (Export)wasm_runtime_validate_app_addr - (Export)wasm_runtime_validate_app_str_addr - (Export)wasm_runtime_validate_native_addr - (Export)wasm_runtime_addr_app_to_native - (Export)wasm_runtime_addr_native_to_app - (Export)wasm_runtime_get_app_addr_range - aot_set_aux_stack - aot_get_aux_stack - wasm_set_aux_stack - wasm_get_aux_stack - aot_check_app_addr_and_convert, wasm_check_app_addr_and_convert and jit_check_app_addr_and_convert - wasm_exec_env_set_aux_stack - wasm_exec_env_get_aux_stack - wasm_cluster_create_thread - wasm_cluster_allocate_aux_stack - wasm_cluster_free_aux_stack ## Data structure changes - WASMModule and AOTModule - field aux_data_end, aux_heap_base and aux_stack_bottom - WASMExecEnv - field aux_stack_boundary and aux_stack_bottom - AOTCompData - field aux_data_end, aux_heap_base and aux_stack_bottom - WASMMemoryInstance(AOTMemoryInstance) - field memory_data_size and change __padding to is_memory64 - WASMModuleInstMemConsumption - field total_size and memories_size - WASMDebugExecutionMemory - field start_offset and current_pos - WASMCluster - field stack_tops ## Components that are affected by the APIs and data structure changes - libc-builtin - libc-emcc - libc-uvwasi - libc-wasi - Python and Go Language Embedding - Interpreter Debug engine - Multi-thread: lib-pthread, wasi-threads and thread manager	2024-03-12 11:38:50 +08:00
Zhen Kong	1a88104160	Remove module instance from hashmap in wasi_nn_destroy (#2613) ... When destroying wasi-nn context, module instance should be also removed from hashmap to avoid memory leak.	2023-10-03 08:33:11 +08:00
tonibofarull	b45d014112	wasi-nn: Improve TPU support (#2447) ... 1. Allow TPU and GPU support at the same time. 2. Add Dockerfile to run example with [Coral USB](https://coral.ai/products/accelerator/).	2023-08-14 20:03:56 +08:00
tonibofarull	0b0af1b3df	wasi-nn: Support uint8 quantized networks (#2433) ... Support (non-full) uint8 quantized networks. Inputs and outputs are still required to be `float`. The (de)quantization is done internally by wasi-nn. Example generated from `quantized_model.py`:  Visualization with [netron](https://netron.app/).	2023-08-11 07:55:40 +08:00
tonibofarull	ab96e01f5e	wasi-nn: Add support of wasi-nn as shared lib (#2310) ... ## Context Currently, WAMR supports compiling iwasm with flag `WAMR_BUILD_WASI_NN`. However, there are scenarios where the user might prefer having it as a shared library. ## Proposed Changes Decouple wasi-nn context management by internally managing the context given a module instance reference.	2023-06-27 18:18:26 +08:00
ayakoakasaka	89be5622a5	wasi-nn: Add external delegation to support several NPU/GPU (#2162) ... Add VX delegation as an external delegation of TFLite, so that several NPU/GPU (from VeriSilicon, NXP, Amlogic) can be controlled via WASI-NN. Test Code can work with the X86 simulator.	2023-05-05 16:29:36 +08:00
tonibofarull	a15a731e12	wasi-nn: Support multiple TFLite models (#2002) ... Remove restrictions: - Only 1 WASM app at a time - Only 1 model at a time - `graph` and `graph-execution-context` are ignored Refer to previous document: e8d718096d/core/iwasm/libraries/wasi-nn/README.md	2023-03-08 15:54:06 +08:00
tonibofarull	1614ce12fa	wasi-nn: Enable GPU support (#1922) ... - Split logic in several dockers - runtime: wasi-nn-cpu and wasi-nn- Nvidia-gpu. - compilation: wasi-nn-compile. Prepare the testing wasm and generates the TFLites. - Implement GPU support for TFLite with Opencl.	2023-02-02 08:09:46 +08:00
tonibofarull	9eed6686df	Refactor WASI-NN to simplify the support for multiple frameworks (#1834) ... - Reorganize the library structure - Use the latest version of `wasi-nn` wit (Oct 25, 2022): 0f77c48ec1/wasi-nn.wit.md - Split logic that converts WASM structs to native structs in a separate file - Simplify addition of new frameworks	2023-01-25 18:32:40 +08:00