d1-omni-600M — native MLX port

Source code and development: Obscyra-app/d1-omni-mlx

Unofficial MLX port of LiquidAI/d1-omni-600M, pinned to 02b55d7076f15129e59ab3f94783f32c4b088674. Same trained weights, no training. Supports text/JSON, text + images, or text + 16 kHz mono audio, with choice, noul (P(yes)) and score questions. Zero generated tokens. Images and audio cannot be combined in a single request. This is a decision model, not a chatbot.

All neural modules run in MLX: bidirectional LFM2 encoder, decision head, SigLIP2 + projector, FastConformer + adapter. Preprocessing uses NumPy, Pillow and SciPy. No PyTorch, Transformers or remote-code execution at inference. The model requires this included runtime; mlx_lm.generate is not its API.

Run on Apple Silicon

With Python 3.12 and a compatible macOS/MLX runtime (tested on macOS 26.6.2):

python3 -m venv .venv
source .venv/bin/activate
pip install huggingface_hub
hf download Nurymanau/d1-omni-600M-MLX-fp16 --revision v0.1.0 --local-dir d1-omni-mlx
cd d1-omni-mlx
pip install -r requirements.txt
python -m d1_omni_mlx --model . --request request.json

from d1_omni_mlx import D1Omni
model = D1Omni('.')
result = model.system_one(
    'I was charged twice. Please refund one payment.',
    {'team': {'type': 'choice',
              'instructions': 'Which team should handle this?',
              'criteria': {'billing': 'Payments and refunds',
                           'technical': 'App failures',
                           'fraud': 'Unauthorized access'}}})
print(result)

Images: pass images=PIL.Image.open(...) or a list. CLI: --image picture.png (repeat for several images). Large images are tiled, following the original.

Audio: pass audio=samples where samples is a one-dimensional float waveform or int16 PCM at 16,000 Hz. CLI: --audio speech.wav; incorrect sample rate is rejected. Clips shorter than 0.5 s are padded; clips over 30 s are truncated, matching the original. Audio understanding was trained on English requests.

Also supported: system_one_batch([(state, questions), ...]), probabilities(state, [question, ...], images=None, audio=None) and probabilities_batch(...). Named questions return original option names. Text temperature calibration and modality-specific prompt formats are preserved. Batch size is bounded to 4,096 padded tokens by default; a longer individual sequence is still allowed. Very long contexts and large image batches can need more memory; the full 16,384-position limit is not a Mac memory guarantee.

Validation scope

The accompanying validation report compares this port against the pinned original Torch FP32 model, using all three question types and modalities, JSON/Unicode/delimiter escaping, different option counts and padded batches. Core suite: 18 cases / 44 questions, plus 9 padded-batch questions. Extended suite: 7 cases / 19 questions, plus 7 mixed-modality-batch questions. All 79 checked decisions (including repeated batch forms) matched the original. FP32 core maximum probability error: 0.0000294. FP16 maximum across all checks: 0.00935212 (0.936 percentage points); zero argmax flips. FP16 is a lossy cast; this is a smoke/parity suite, not an accuracy benchmark or proof of identical behavior on every input. See VALIDATION.md and validation.json in the built package for final results.

FP32 reference weights occupy about 2.35 GB; FP16 about 1.17 GB. BF16 is not recommended by the original model authors. No quantized 4-bit/8-bit claim. Preprocessing follows the pinned Torch ARM CPU numerical behavior, including uint8 antialiased resize and silence normalization. Other Torch backends may have slightly different rounding.

License and provenance

LFM Open License v1.0; read the included LICENSE and NOTICE. This port does not change the original license. Dependencies/algorithm attributions are retained in licenses/. manifest.json records every shipped file's SHA-256 and the original checkpoint SHA. No endorsement by Liquid AI or Apple is implied.