An experimental Apple Silicon MLX conversion of NVIDIA's Dory 1.0 4B BF16 model, a looped reasoning model. The source inventory has approximately 3.975 billion parameters; the converted file has 655 tensors, with 146 quantized modules identified on reload.
Runtime support is required. Stock oMLX 0.7.3 lacks the Dory MLX-LM implementation. The relevant MLX-LM Dory PR #1988 was pending in the recorded environment; do not assume it has merged or that a stock installation supports this model. The recorded execution used a Dory-enabled MLX-LM overlay and a Dory-specific tokenizer fallback. Support source and patches are provided for two alternative, pinned integration routes; they are not a full runtime. trust_remote_code does not replace that runtime.
The stored configuration defaults to two recurrent loops and per_loop attention caches. Weights are shared across loops, but attention caches are distinct per loop. The configuration declares 262,144 positions; usable context depends on available memory and runtime. The model's template defaults to reasoning mode, which may consume the output budget before a final answer.
model.safetensors and its index: converted weights and tensor map.config.json, tokenizer files, chat_template.jinja and generation_config.json: model and text interface metadata retained from the conversion/source.SOURCE_MODEL_README.md: original NVIDIA model card, preserved unchanged. Its relative image and ethical/safety subcard files are not included here; consult the original NVIDIA card for upstream assets and safety information. Its BF16 instructions and claims do not automatically apply to this MLX conversion.LICENSE, LICENSE-OPENMDW-1.1.txt, LICENSE-MLX-LM.txt: model agreement, retained notice and MLX-LM code license.provenance.json, EVALUATION.md, METHODS.md and figures/: recorded conversion, selected local observations and methods.support/: pinned MLX-LM architecture and tokenizer patch, plus a separate oMLX source patch; see integration notes.This is uniform MLX affine 8-bit quantization with group size 64 for eligible modules, with remaining parameters in BF16. It is not mixed-precision calibration, importance-matrix quantization, or an oQ format. It does not specify KV-cache quantization. A strict fresh reload identified 146 affine quantized linear/embedding modules. See provenance.json for source snapshot, revisions and hashes.
The recorded conversion used Python 3.11.18, MLX 0.32.4, MLX-LM 0.32.3.dev8, and Transformers 5.20.2. The API call was:
from mlx_lm.convert import convert
convert(source_path, mlx_path=output_path, quantize=True,
q_group_size=64, q_bits=8, q_mode="affine",
dtype="bfloat16", trust_remote_code=False)
Use an Apple Silicon machine with sufficient unified memory and a separately installed, compatible Dory-enabled MLX-LM runtime. Point that runtime at the local directory; for example, with the recorded overlay already installed and active:
MLX_ENABLE_TF32=0 python -m mlx_lm.generate \
--model /path/to/NVIDIA-Nemotron-Labs-Dory1.0-4B-MLX-8bit-g64 \
--prompt 'Explain what a recurrent language model is.' --max-tokens 256
This is an illustrative direct MLX-LM CLI invocation, not a new test of that command. Check the installed CLI and Dory support first. Prompt formatting, generation settings and memory requirements depend on the runtime. The 256-token bound can truncate reasoning before an answer.
A locally patched oMLX 0.7.3 installation made one successful cold, nonstreaming two-loop chat request with this weight package. That observation does not establish stock oMLX support or full serving compatibility. The separate four-loop GPQA and context experiments used direct MLX with an in-memory model configuration override, not a demonstrated oMLX per-request loop flag. This package does not promise a four-loop oMLX switch.
The same Q8 weights were evaluated with greedy decoding on 198 questions; unanswered items counted as incorrect. The local scores are roughly comparable to the BF16 scores in NVIDIA's model card:
| Loops | NVIDIA BF16 reported | Local MLX Q8 |
|---|---|---|
| 2 | 72.41% | 138/198 = 69.70% |
| 4 | 72.16% | 149/198 = 75.25% |
The evaluation methods were not matched, and these results do not establish numerical parity or isolate quantization effects. See EVALUATION.md and METHODS.md for results, methods and limitations. The benchmark runner and complete setup are not published, so exact replay is unavailable from this package.
This is experimental. No matched full BF16/Q8 numerical parity study, exhaustive finite-weight scan, or independent real-weight reference parity was established. Effective complete EOS behavior in oMLX, prefix-cache hits/persistence correctness, tools, streaming, concurrency and long-context server behavior were not established by the one live smoke. The runtime emitted Dory-type and tokenizer-regex warnings; no speculative regex change was applied. Memory observations are sampled rather than physical allocation caps. See EVALUATION.md.
The NVIDIA model is governed by the OpenMDW-1.1 agreement, reproduced in LICENSE. Distribution requires retaining the agreement and applicable copyright and origin notices; see its full terms. The source NVIDIA model card and upstream MLX-LM MIT notice are included. Dory implementation credits and revision are recorded in provenance.json. This summary is not legal advice.