Qwimi3.5-9B KimiK2.6 Opus Distill MLX 8-bit

This is the MLX 8-bit conversion of:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

It is intended for Apple Silicon / MLX users who want to run the Qwimi3.5-9B reasoning fine-tune locally.

This repo is an MLX-format conversion/quantization of the merged BF16 model. The training was done separately using Unsloth + LoRA SFT, then merged to BF16 before MLX conversion.

---

Model Summary

---

Quantization Notes

This repository contains the 8-bit MLX version.

The 8-bit version is the highest-quality MLX quantization target in this set. It uses more storage and memory than 4-bit or 6-bit, but should generally preserve the most behavior from the merged BF16 source model.

For other formats, see the main release family:

• BF16: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16
• GGUF: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-GGUF

---

Install

pip install -U mlx-lm

---

Basic Usage

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit \
  --prompt "Explain RS-LoRA in simple terms."

For longer generations:

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit \
  --prompt "Solve this step by step: If a company grows revenue from 120M to 180M over 3 years, what is the CAGR?" \
  --max-tokens 1024

---

Chat Template / Prompting

This model was trained with Qwen-style chat formatting and response-only SFT. For best results, use an instruction/chat style prompt.

Example:

You are a helpful reasoning assistant.

Question:
Explain why RS-LoRA can help at higher LoRA ranks.

The model was trained on reasoning-style outputs where assistant responses often contain:

<think>
reasoning trace
</think>
final answer

Depending on your inference settings and prompt, the model may produce explicit reasoning traces. For end-user applications, you may want to post-process or hide internal reasoning and show only the final answer.

---

Training Data

The merged BF16 source model was trained on a combined reasoning/distillation mix from:

1. trjxter/Kimi-K2.6-Reasoning-3300x-WandB
2. Jackrong/Qwen3.5-reasoning-700x
3. Jackrong/Claude-opus-4.6-TraceInversion-9000x

After formatting and 16k-token filtering, the final usable dataset size was:

12,366 usable examples
12,000 train examples
366 held-out eval examples
200 trainer eval examples

The curation process normalized mixed schemas into a shared Qwen chat-template SFT format.

---

Training Configuration

The original fine-tune used:

Base model: unsloth/Qwen3.5-9B
Training: LoRA SFT
Base loading: 8-bit
LoRA rank: 128
LoRA alpha: 128
RS-LoRA: enabled
Max sequence length: 16,384
Gradient checkpointing: unsloth
Batch size: 2
Gradient accumulation: 8
Effective batch size: 16
Epochs: 1
Optimizer: adamw_8bit
Learning rate: 2e-5
Warmup steps: 30
Save steps: 200
Eval steps: 200

Expected optimizer steps:

12,000 / 16 = 750 optimizer steps

---

Training Results

Final trainer metrics from the source fine-tune:

{
    "train_runtime": 15728.7981,
    "train_samples_per_second": 0.763,
    "train_steps_per_second": 0.048,
    "total_flos": 1.4528477736160543e+18,
    "train_loss": 0.5516796213785807,
    "epoch": 1.0,
}

Loss checkpoints:

---

Artifact Lineage

flowchart TD
    A[unsloth/Qwen3.5-9B] --> B[LoRA SFT]
    B --> C[Qwimi LoRA adapter]
    C --> D[Merged BF16 model]
    D --> E[MLX 8-bit conversion]

---

Intended Use

This model is intended for:

• general reasoning experiments,
• local Apple Silicon inference,
• MLX benchmarking,
• educational fine-tuning research,
• testing Qwen-style reasoning distillation behavior.

It is not intended for high-stakes medical, legal, financial, or safety-critical decision-making without expert review.

---

Limitations

• This is a community fine-tune and should be evaluated before production use.
• The model may produce incorrect reasoning or confident wrong answers.
• The model may generate verbose reasoning traces depending on prompts.
• MLX quantized behavior can differ from BF16 or GGUF behavior.
• This README reports the training metrics of the BF16 source model, not a separate full eval of this exact MLX quant.

---

Citation / Attribution

Base model and tooling credit goes to the Qwen, Unsloth, Hugging Face, and MLX communities.

Source fine-tune:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

MLX conversion:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit

---

Notes

This MLX quant was created to make the Qwimi3.5-9B run easier to use for Apple Silicon users.