Ling 3.0 Flash — oQ5

A native Apple-silicon conversion of inclusionAI/Ling-3.0-flash, quantized with oMLX's optimized mixed-precision oQ5 recipe and packaged for oMLX and compatible MLX-LM runtimes.

Original model · InclusionAI on ModelScope · OpenRouter · MLX-LM

About this conversion

This repository contains an oMLX-optimized mixed-precision oQ5 conversion of Ling 3.0 Flash. Ling is a 124B-total / 5.1B-active hybrid-linear mixture-of-experts model. The conversion preserves the upstream tokenizer and chat template and includes the custom MLX architecture adapter required for Kimi Delta Attention, gated MLA, and sparse MoE layers.

[!IMPORTANT] This model uses the included bailing_hybrid.py custom MLX adapter. In oMLX, enable Trust Remote Code for this model before loading it. Standalone MLX-LM requires a build whose mlx_lm.load supports the trust_remote_code argument and repository-provided model_file adapters; stock PyPI mlx-lm 0.31.3 does not provide that loader path.

Apple-silicon performance

This checkpoint was load-tested, generation-tested, and benchmarked on:

A warmed local test produced:

The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance.

Recommended use with oMLX

1. Download the model into the oMLX model directory:

hf download Vontra/Ling-3.0-flash-oQ5 \
  --local-dir ~/.omlx/models/Vontra/Ling-3.0-flash-oQ5

2. Refresh the oMLX model registry.
3. Open the model settings and enable Trust Remote Code.
4. Load Ling-3.0-flash-oQ5 and use the normal chat UI or OpenAI-compatible endpoint.

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "Ling-3.0-flash-oQ5",
    "messages": [{"role": "user", "content": "Explain why hybrid linear attention is useful."}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 512
  }'

Thinking mode is enabled by the upstream chat template by default. Disable it with the model's chat-template configuration when a direct answer is preferred.

Compatible standalone MLX-LM builds

With an MLX-LM build that supports repository-provided model adapters:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-oQ5 \
  --trust-remote-code \
  --prompt "Explain why hybrid linear attention is useful." \
  --max-tokens 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

Python usage with the same capability:

from mlx_lm import load, generate


model, tokenizer = load(
    "Vontra/Ling-3.0-flash-oQ5",
    trust_remote_code=True,
    tokenizer_config={"trust_remote_code": True},
)
messages = [
    {"role": "user", "content": "Explain sparse mixture-of-experts routing."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Only enable remote code after reviewing the included adapter files.

Architecture

Ling 3.0 Flash alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts.

The included MLX adapter uses MLX-LM primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is excluded from ordinary causal generation; this release does not claim MTP or DSpark speculative-decoding support.

Upstream model highlights

InclusionAI describes Ling 3.0 Flash as a hybrid reasoning model for software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding.

The upstream defaults are:

thinking: enabled
temperature: 0.6
top_p: 0.95
top_k: 20

For benchmark methodology, scores, intended use, limitations, and framework-specific deployment instructions, see the original InclusionAI model card.

Conversion and validation notes

• Source weights: upstream BF16 checkpoint.
• Quantization mode: affine, group size 64.
• oQ5 uses a 5-bit base with 455 sensitive modules kept at higher precision.
• Both quantization and quantization_config preserve the per-module recipe.
• The upstream chat template is included unchanged.
• All 2,067 converted tensors and all 16 indexed shards were checked locally.
• The custom adapter was loaded with explicit trust and exercised through end-to-end generation.
• Quantization can reduce output quality relative to BF16; use a higher-precision variant when quality matters more than memory use.

This is a community conversion, not an official InclusionAI release. Validate quality and numerical behavior on representative workloads before production use.

License and attribution

The upstream model declares the MIT License in its Hugging Face metadata. A standard MIT licence copy is included in this repository.

All model design, training, benchmark, and upstream documentation credit belongs to InclusionAI and the original contributors. The mixed-precision conversion, Apple-silicon validation, compatibility packaging, and model card are provided by Vontra.

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

Published peak memory: 87.44 GB; estimated starting tier: 128GB, leaving about 40 GB nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download Vontra/Ling-3.0-flash-oQ5 --local-dir ./models/Ling-3.0-flash-oQ5

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow Vontra for new Apple Silicon releases and fixes.