CYBER-FROST-3.8-MLX-4bit

CYBER-FROST-3.8, Blackfrost-AI's model for security professionals doing authorized research, assessment, engineering and response work, in a high-quality 4-bit MLX build for Apple Silicon. Tuned for long agentic contexts with thinking, and built to run as fast as possible with TensorFold: 96 tok/s at 26k context, with a retrained MTP draft head.

Made in collaboration with Blackfrost-AI. oQ4e mixed precision plus layer-by-layer distillation against the full BF16 model: 34% closer to BF16 than a standard oQ4e conversion at the same size and speed. Vision tower included.

Lineage: Qwen/Qwen3.8-Flash-Next → Blackfrost-AI/CYBER-FROST-3.8-BF16 (security fine-tune) → this build (oQ4e mixed precision, then layer-wise DWQ against the BF16 weights).

The MTP (multi-token-prediction) draft head and the vision tower are both kept and working.

* measured the previous day with the stock template and 5 warm runs; the two builds share the same engine path, so the difference is draft acceptance, not raw speed.

⚠️ Intended use and terms

CYBER-FROST is built by Blackfrost-AI for qualified security professionals doing authorized work: defensive research, secure code review, vulnerability validation, red/purple-team exercises, detection engineering, incident response, malware analysis and controlled security-agent workflows. This quantization inherits those terms.

It must not be used for anything illegal, in any jurisdiction, under any circumstances, including accessing systems or data without permission, deploying malware, stealing credentials or data, or disrupting services. The model reduces false refusals on security topics by design; it is not a safety control, an authorization service, a sandbox or a policy engine. Authorization, scope, credentials, network access and human approval must be enforced outside the model. Generated findings, code and commands may be wrong or fabricated: review them and run them only in isolated, authorized environments. You are solely responsible for your use of it.

---

Quality: DWQ vs a standard oQ4e conversion

Both quantized builds below use the same 4/5/6/8-bit mixed-precision layout (oQ4e) and the same file size; they differ only in the quantization scales. "This release" is oQ4e + DWQ. Closeness to the BF16 original is measured by running BF16 itself (streamed one layer at a time) side by side with each build on the same tokens: KL = KL(BF16 ‖ build) per token, lower = closer; agreement = how often the build's most likely next token is BF16's.

Held-out general text (260,096 tokens of code / chat / tool-use / reasoning / multilingual text that neither the DWQ training nor the iMatrix calibration used):

This release is closer to BF16 on 252 of 254 sequences and in all 10 text categories (95% CI of the KL difference −0.0241 to −0.0199).

Fixed text sets (12,288 tokens of Python-documentation prose, standard-library code and worked reasoning, plus 7,168 tokens of security documentation: OpenSSH / OpenSSL / pf / codesign manuals and Python's ssl, hashlib, hmac, secrets documentation):

Long agentic context (one real 26,161-token coding-agent conversation, scored at every position): on the model's own turns, both builds stay very close to BF16 all the way to 26k tokens (98.5–99.7% agreement, no drift with context length), and this release is closer than oQ4e (KL 0.0156 vs 0.0213). On tool output (HTML, file listings), where even BF16 cannot predict the text, this release's lead over oQ4e is largest.

Performance

Measured on an M3 Ultra Mac Studio (256 GB) with TensorFold 0.6.5 (MLX 0.32.2), serving this checkpoint as-is.

Production agentic workload: a real coding-agent conversation (70 agent turns plus 4 tool schemas; 26,235 tokens with this release's default CYBER-FROST template), 1,024-token replies, temperature 1.0 / top-p 0.95 / top-k 20, reasoning effort xhigh, MTP drafting on. Cold = fresh server, nothing cached. Warm = the same prompt again (prefix cached), mean of 8. Agent turn = the conversation plus one tool call and its 4,171-token tool output (median of 3).

TensorFold's own short-reply bench (tools/bench_openai.py, --no-thinking, 64-token replies, median of 5), decode tok/s:

Short replies decode faster than the long agentic workload because drafts are accepted more often on predictable text; greedy (temperature 0) figures depend on the single reply each prompt produces.

Every drafted reply is token-for-token identical to serial decoding (TensorFold's drafted-vs-serial check, 9/9 cases: prose, code and the long agent prompt; sampled and greedy).

Chat template and persona

Ships with CYBER-FROST's own chat template as the default (chat_template.jinja): it supplies Blackfrost-AI's default operating prompt, accepts caller-provided system context, turns thinking on by default, and takes reasoning_effort xhigh (default), medium or low.

The stock Qwen3.8-Flash-Next template is included as an alternative (no injected prompt; thinking can be turned off with enable_thinking: false). Switch with the bundled script from the model directory, then restart the server:

bash switch_template.sh qwen         # stock Qwen template
bash switch_template.sh cyberfrost   # CYBER-FROST template (default)
bash switch_template.sh status

The template lives in chat_template.jinja and in the chat_template key of tokenizer_config.json; the script switches both (pairs: chat_template_cyberfrost.jinja · tokenizer_config_cyberfrost.json and chat_template_qwen.jinja · tokenizer_config_qwen.json). It replaces symlinks rather than writing through them, so an hf download cache checkout stays intact. Weights are identical either way.

The production figures above were measured with the default CYBER-FROST template; its operating prompt adds about 70 tokens to every request (cached after the first turn).

Quick start

TensorFold (text, MTP)

TensorFold version: run this build with TensorFold 0.6.5 for now. TensorFold 1.0's new native engine does not read mixed-precision builds like this one yet; support is expected in TensorFold 1.0+ shortly.

hf download grant-ai/CYBER-FROST-3.8-MLX-4bit --local-dir ~/models/CYBER-FROST-3.8-MLX-4bit
tensorfold serve ~/models/CYBER-FROST-3.8-MLX-4bit --reasoning-effort xhigh --context 40960

MTP drafting is on by default (--mtp-drafts 3 on Macs); --no-thinking turns reasoning off. TensorFold serves Flash Next images on CUDA only; use oMLX for image input on a Mac.

oMLX (text, vision, MTP)

omlx serve --model-dir ~/models --port 8000

with mtp_enabled: true and mtp_num_draft_tokens: 3 for this model in oMLX's model settings.

Both expose an OpenAI-compatible endpoint:

curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "CYBER-FROST-3.8-MLX-4bit",
  "messages": [{"role": "user", "content": "Review this nginx config for SSRF exposure: ..."}],
  "max_tokens": 4096, "temperature": 1.0, "top_p": 0.95, "top_k": 20}'

Settings that matter

Technical details

How it was made

1. Source: Blackfrost-AI/CYBER-FROST-3.8-BF16 at revision 3e30d25, every file size-checked against the Hub.
2. oQ4e with oMLX's quantizer at group size 32, iMatrix-enhanced. The iMatrix was collected on the real BF16 weights, streamed one decoder layer at a time (no 4-bit stand-in model), on 256 sequences × 1,024 tokens of oMLX's built-in code / chat / tool-use / reasoning / multilingual corpus. MTP head and vision tower preserved.
3. Layer-wise DWQ (distilled weight quantization): for each of the 48 decoder blocks in order, the BF16 block is run on the BF16 activations and the quantized block on the quantized model's own activations, and only the quantized block's scales and biases are trained so its output matches the BF16 block's. Packed 4/5/6/8-bit weights, norms, routers, embeddings and the vision tower are byte-identical to the oQ4e build, so size, layout and speed do not change. A block's update is kept only if it lowers error on held-out sequences. The last block (layer 47), whose output feeds the final norm and output head, is trained instead to match BF16's next-token distribution (KL), which improved closeness to BF16 where plain output matching did not.
4. MTP draft head retrained. The upstream head predates the parent's final training stage. It was distilled (starting from the BF16 head) to predict this release's own next-token distribution under the production sampler (temperature 1, top-k 20, top-p 0.95), on long agentic coding conversations and general coding, chat, reasoning and tool-use prompts with replies written by this model, then quantized back to the exact formats the head had (no size or layout change). The head only proposes drafts; TensorFold keeps a draft only if it equals the token the model itself produces, so output is unchanged and only speed improves.
5. Verification: byte-level diffs at each stage (DWQ changed only scale/bias slots; the head retrain changed only MTP-head tensors), drafted-vs-serial exactness in TensorFold, the quality and speed measurements above, and real image requests through oMLX (a synthetic image with known counts and text, read back exactly, and a photograph described correctly).

Limitations

• No standardized cyber-capability benchmark has been run on this build (none was published for the BF16 parent either). Quality here means closeness to the BF16 model and text likelihood, not task competence.
• Refusal behaviour was not measured on this build. Blackfrost-AI describes the parent as reducing false refusals in authorized security work; quantization can shift that.
• The vision tower is carried through and answers image questions correctly in the checks above, but Blackfrost-AI has not released a multimodal quality evaluation for the parent.
• Blackfrost-AI notes the upstream MTP head predates the parent's final behavioural stage; this release ships a retrained head: on the long agentic workload at temperature 1 it produces 9.7% more tokens per draft step than the upstream head (same prompts, byte-identical replies). It was trained on general coding, chat, reasoning, tool-use and agentic material; security-specific contexts were not separately measured.
• Speed depends on machine, context length, reasoning effort and template.

Credits

• Qwen for Qwen3.8-Flash-Next.
• Blackfrost-AI for CYBER-FROST-3.8 and its security corpus.
• oMLX for the oQ mixed-precision quantizer; TensorFold for the inference engine.

Disclaimer and licence

This conversion was made in collaboration with Blackfrost-AI, the authors of CYBER-FROST-3.8. It is not endorsed by, sponsored by or associated with Qwen, Alibaba Cloud, oMLX or TensorFold, none of which reviewed or contributed to this repository or bear any responsibility for it.

Changes made to the upstream work: quantized to oQ4e mixed precision with an iMatrix from the BF16 weights; scales and biases refined by layer-wise DWQ against the BF16 weights; MTP draft head retrained on this build's own outputs; vision tower preserved; the stock Qwen chat template and a template-switch script added alongside the upstream template, which remains the default. Upstream deployment kit, harness results and image assets are not included; see the parent repository.

Licence: Qwen Community License 1.0 (LICENSE, inherited from the upstream model and weights). Your use is also subject to the terms of the upstream works.