all-MiniLM-L6-v2 Text Embeddings using WebNN.
This model has been created with webnn-graph (https://github.com/tarekziade/webnn-graph), using
https://huggingface.co/Xenova/all-MiniLM-L6-v2 ONNX files as a source.
Lowering this BERT model from ONNX to WebNN was not a straightforward task because ONNX allows dynamic graphs unlike WebNN and lowering the dynamic graph to a static one that can be executed in WebNN requires a complex flow. Read it up here https://blog.ziade.org/webnn-graph/onnx-lowering.html
The resulting WebNN graph is composed of 3 files:
model.webnn: Graph definitionmodel.weights: Model weightsmanifest.json: metadata, describes tensor shapes, data types and other things
To run this model, install PyWebNN and the transformers stack
pip install pywebnn transformers torch onnxruntime
Then, grab the demo script here : https://github.com/tarekziade/rustnn/blob/main/examples/minilm_embeddings.py
That script will create embeddings for 5 sentences using the original sentence transformer and then using WebNN, and make sure they are both producing the same embeddings.
Then run the demo:
➜ python demo.py
======================================================================
all-MiniLM-L6-v2 Text Embeddings Demo
======================================================================
----------------------------------------------------------------------
Initializing Transformers (Reference) Implementation
----------------------------------------------------------------------
[INFO] Loading transformers model: sentence-transformers/all-MiniLM-L6-v2
[OK] Transformers model loaded
----------------------------------------------------------------------
Encoding with Transformers
----------------------------------------------------------------------
Encoding 5 sentences...
[OK] Generated 5 embeddings
[OK] Shape: (5, 384)
[OK] First embedding (first 10 dims): [ 0.00021132 0.06401434 -0.01092573 0.01032474 -0.00429077 0.02909725
0.04863852 -0.04892546 0.02078866 -0.02798944]
[OK] L2 norm: 1.000000
----------------------------------------------------------------------
Initializing WebNN Implementation
----------------------------------------------------------------------
[INFO] Loading tokenizer...
[INFO] Loading WebNN model from tarekziade/all-MiniLM-L6-v2-webnn...
[INFO] Downloading model from Hugging Face Hub: tarekziade/all-MiniLM-L6-v2-webnn
[CACHED] model.webnn
[CACHED] model.weights
[CACHED] manifest.json
[OK] Model downloaded
[INFO] Loading graph...
[OK] Graph loaded successfully
- Operand count: 742
- Operation count: 445
- Inputs: ['attention_mask', 'input_ids', 'token_type_ids']
- Outputs: ['last_hidden_state']
----------------------------------------------------------------------
Encoding with WebNN
----------------------------------------------------------------------
Encoding 5 sentences...
[OK] Generated 5 embeddings
[OK] Shape: (5, 384)
[OK] First embedding (first 10 dims): [ 0.00021128 0.06401427 -0.01092580 0.01032469 -0.00429083 0.02909716
0.04863857 -0.04892550 0.02078862 -0.02798957]
[OK] L2 norm: 1.000000
======================================================================
[OK] Embeddings are VERY SIMILAR (cosine > 0.99)
======================================================================
[OK] Done. Cache dir: /Users/tarekziade/.cache/webnn/