Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-oQ4e-fp16-mtp
This repository is a complete Apple MLX deployment of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, converted with oMLX v0.6.2 using importance-matrix-enhanced oQ mixed-precision quantization.
Download the whole repository: the Safetensors shards require the included index, model config, tokenizer, chat template, and image/video processor files. This is not a GGUF, Transformers, or NInfer artifact.
Quick facts
| Item | Value |
|---|---|
| Model type | qwen3_5 |
| Quantization layout | Affine Q4/G64 by default, with 166 Q5/G64 tensor overrides. |
| Tensor payload | 17,893,140,142 bytes / 16.66 GiB |
| Safetensors shards | 4 |
| Conversion runtime | oMLX 0.6.2 |
| Calibration | oqe_code_multilingual, 128 samples × 512 tokens |
| Included model features | Vision resources and one MTP layer |
| Intended runtime | oMLX on Apple Silicon/macOS |
About the FP16 suffix: this is an M1/M2-oriented compatibility layout, not a fully FP16 MTP head. Safetensors headers show 22 auxiliary/non-quantized MTP tensors (including metadata, norms, scales, and biases) stored as F16, while seven major MTP projection weights remain packed Q4/U32.
Choose a variant
| Variant | Nominal tier | Tensor payload | Shards |
|---|---|---|---|
| oQ4e + FP16 MTP auxiliaries (this repo) | 4-bit | 17,893,140,142 bytes / 16.66 GiB | 4 |
| oQ4e | 4-bit | 16,971,681,558 bytes / 15.81 GiB | 4 |
| oQ6e | 6-bit | 23,716,288,460 bytes / 22.09 GiB | 5 |
| oQ8e | 8-bit | 30,001,641,934 bytes / 27.94 GiB | 6 |
These tiers differ in storage and quantization layout. No same-Mac quality, memory, TTFT, or throughput comparison is published here, so the table should not be read as a benchmark.
Download
Install the Hugging Face CLI, then place the complete repository below oMLX's model directory:
mkdir -p "$HOME/.omlx/models/pyros-vault"
hf download pyros-vault/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-oQ4e-fp16-mtp \
--local-dir "$HOME/.omlx/models/pyros-vault/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-oQ4e-fp16-mtp"
Serve with oMLX
Install the current oMLX runtime and start its OpenAI-compatible server:
brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
omlx serve --model-dir "$HOME/.omlx/models"
Discover the exact model ID exposed by your installed oMLX version:
curl http://127.0.0.1:8000/v1/models
Use that returned ID with the OpenAI-compatible endpoint. MTP files being present does not automatically enable speculative decoding: Lightning MTP is opt-in through oMLX model settings, and behavior can vary by runtime version and Apple chip.
Quantization and verification
The bundled oq_imatrix_report.json records calibration with oqe_code_multilingual over 128 sequences of 512 tokens. The included report records 504 importance entries, 503 applied modules, two missing names, and no shape mismatches.
The report and tensor metadata establish how the artifact was built; they are not an end-to-end quality benchmark. Repository structure, configs, shard counts, payload sizes, and quantization metadata were audited for this card. Inference was not rerun on a Mac, so no local speed, memory, MTP-acceptance, Vision-quality, or long-context claim is made.
Provenance
This is a deployment conversion of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1. The source card's behavior, benchmark, and training claims remain upstream claims and were not independently reproduced here. The repository retains the source model's Vision resources and one MTP layer. It does not contain NInfer DFlash weights.
Limitations
- MLX/oMLX targets Apple Silicon and macOS; this repository is not runnable through CUDA on Windows.
- Hugging Face hosted inference does not serve this custom oMLX layout.
- The config advertises a 262,144-token maximum context. That value is model metadata, not a claim that this full context was tested or will fit your machine.
- Vision preprocessing, tool use, MTP acceptance, memory use, and throughput depend on the oMLX version, client, prompt, and Apple hardware.
- Quantization can change output quality. Evaluate this exact variant on your workload.
License and credits
The direct upstream declares Apache-2.0. Review its model card and repository files for the full attribution and usage terms.
Quantized and packaged by pyros-vault with oMLX/oQ.