Qwen3-0.6B-lk-alpha-20k-MNN

Published by TokForge for on-device inference.

Qwen3-0.6B draft model exported for TokForge + MNN speculative decoding, trained with an LK Alpha objective instead of standard KL.

This bundle is aimed at people who want to experiment with acceptance-oriented draft training on mobile.

Why this repo exists

This is the LK-loss variant of our 20K Qwen3 draft lane:

Qwen3-0.6B student
Qwen3-8B teacher
20K teacher dataset
LK Alpha training objective
exported as a ready-to-use MNN draft bundle

Best-known use

Draft model backend: CPU
Draft threads: 2
Draft predict length: d=3
Target pairing: usually Qwen3-8B in TokForge

Benchmark notes

We do not publish per-device numbers for this experimental variant. Results vary by device, target model, and workload. For measured results on the current default draft line, see TokForge-AccelerationPack-Draft.

Training acceptance (alpha) at the final logged epoch:

0.7350

Included files

llm.mnn
llm.mnn.weight
llm_config.json
config.json
config_cpu.json
tokenizer files
ONNX export artifact for reference

Usage

This bundle is meant for TokForge / MNN, not standard HF Inference.

Typical TokForge recipe:

{
  "backend_type": "opencl",
  "thread_num": 4,
  "precision": "low",
  "memory": "low",
  "sampler_type": "greedy",
  "speculative_type": "draftmodel",
  "draft_predict_length": 3,
  "draft_config_path": "/path/to/config_cpu.json"
}

Known-good draft-side config:

{
  "backend_type": "cpu",
  "thread_num": 2,
  "precision": "low",
  "memory": "low",
  "sampler_type": "greedy"
}

Notes

In our testing, this trained objective improved acceptance over the KL baseline.
On short device benchmarks, the runtime win was in the same general band as the KL model.
This makes it a good experimental alternative, but not a guaranteed universal replacement.

Limitations and Intended Use

Intended for speculative decoding with larger Qwen3 targets inside TokForge.
Training acceptance improved over the KL baseline, but device throughput gains stayed in a similar band on short runs.
Best current evidence is strongest on Qwen3-8B.
This is a specialized runtime artifact, not a general-purpose pretrained release.

Collection

TokForge Mobile Draft Models

TokForge

Website: https://tokforge.ai
Discord: https://discord.gg/Acv3CBtfVm
Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

If you benchmark this on your own device, feel free to share results in Discord.

Credits

Qwen3-0.6B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
Qwen3-8B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license. Used as the distillation teacher.
Converted with llmexport from MNN by Alibaba (Apache-2.0).

darkmaniac7/Qwen3-0.6B-lk-alpha-20k-MNN