加载中…
加载中…
A 0.92B-parameter image decision model for Apple silicon, distilled from Cloudflare's
Clef-flash (Qwen3.5-9B) into a
Qwen3.5-0.8B backbone with a copy of Clef's joint schema head.
Same contract as Clef and the Jev/SystemOne API: a state (text or JSON), optional images, and typed questions
(noul yes/no, choice, ordered score) go in; one probability for every allowed option of every question
comes out of a single forward pass. Nothing is generated.
Runtime: ClefVisionManager in FluidUse (Swift, macOS 15+).
Training, evaluation and conversion code: Fluid Inference model-lab (models/clef-vision-0.8b); conversion
scripts also in mobius.
| Package | Role | Size | Precision |
|---|---|---|---|
Vision_P784/VisionTower_fp32.mlpackage |
Qwen3.5 vision tower, one image ≤ 784 patches (448² cap) → 196 tokens | 377 MB | fp32 (fp16 and mixed precision are numerically wrong on this ViT's activation range) |
LM_L512, LM_L1024, LM_L2048/LMRows_fp16.mlpackage |
Qwen3.5-0.8B decoder + final norm, prefill only, one bucket per sequence length | ~960 MB each | fp16 |
Head_L1024_Q16_O64/Head_fp32.mlpackage |
joint schema head: ≤ 16 questions, ≤ 64 options, one logit per option | 264 MB | fp32 |
embeddings.f16 |
tied token embedding table (host-side gather, also the head's lexical rows) | 508 MB | fp16 |
The host does the gathers between packages: tokenization and Clef's record rendering, image smart-resize /
bicubic / patchify, learned position-embedding interpolation and 2-D rope for the vision tower, token embedding
gather with the vision tokens spliced in, M-RoPE tables, and the head's span-mean matrices. config.json lists
every shape and constant; the Swift runtime is the reference host.
Held-out test, 1,500 records / 4,094 questions over Oxford-IIIT Pets, Food-101 and COCO val images (classification, reference matching, tile grids), PyTorch student:
| Teacher agreement | Gold accuracy | KL | |
|---|---|---|---|
| this model (0.8B) | 92.5% | 92.2% | 0.095 |
| teacher Clef-flash 9B (MLX 8-bit) | — | 94.1% | — |
Core ML pipeline vs the PyTorch student on held-out records: max logit difference 6.3e-3, probability drift 2.3e-4, argmax agreement 100%.
Per task (gold): COCO object presence 98%, Food-101 dish 97%, pets breed 95%, reference match grids 96–100%, tile grids 88–93%. "Select all tiles with cats / dogs" 3×3 grids scored one tile per record: 90% whole-grid pass, 98.9% tile accuracy.
CPU_AND_GPU)| Stage | Median |
|---|---|
| vision tower, one image | 35–51 ms |
| LM rows, 1,024 tokens | 190 ms |
| head | 14 ms |
| one record, 1 image, ~500 tokens | ≈ 260 ms from Python (coremltools), 136–157 ms from the Swift runtime (vision 39, LM 95–113, head 5 ms); PyTorch MPS: 604 ms |
| 4-image / 1,111-token grid, Swift runtime | 640 ms |
The Swift runtime (FluidUse ClefVisionCheck parity) matches the PyTorch student on the bundled fixtures to a
max probability drift of 8.2e-3 with identical argmax. The Qwen3.5 decoder does not run on the Neural Engine (same
as Kev / Cua-S1); the vision tower is fp32, so this is a GPU model.
14,246 records (train 5,648 / dev 2,167 / test 6,431, hash-split by image) labelled by Clef-flash running locally in MLX (8-bit backbone, exact MLX port of the head; validated at ARC-Easy 100% and BANKING77 96.7% against Cloudflare's published 99.5 / 90.9). Student: LoRA r64 on the language model + full head, loss = KL(T=2) to the teacher + 0.3 × label-smoothed CE on dataset gold, 530 steps × 16 records, batch 1 with gradient checkpointing on a 24 GB M5 Pro (6.2 h). Vision tower frozen.
Apache-2.0 (this model, Qwen3.5-0.8B, Clef). Training images: Oxford-IIIT Pet (CC BY-SA 4.0), Food-101 (research use), COCO 2017 val (CC BY 4.0).
Qwen/Qwen3.5-0.8B