Jasper Token Compression 600M — ONNX INT-8
ONNX export of infgrad/Jasper-Token-Compression-600M.
Precision: INT8 (Dynamic)
Quantization: Dynamic INT8
Model size: 583.56 MiB
Dynamic INT8 ONNX export optimized for fast CPU inference. This is a text embedding model.
Benchmarks
| Tokens | Median latency | Tokens/s |
|---|---|---|
| 32 | 44.362 ms | 721.3 |
| 128 | 48.609 ms | 2,633.3 |
| 512 | 63.180 ms | 8,103.8 |
| 1024 | 84.619 ms | 12,101.3 |
Fidelity
Median cosine similarity versus FP32: ~0.988–0.992.
Attribution
Original model: infgrad/Jasper-Token-Compression-600M