Sign up

xy10086/Qwen3.8-9B-Distill-Uncensored-Heretic-MTPLX-Q6G64

🤗 Hugging Face sourceimage-text-to-textapache-2.09.4B params19 GBsafetensors✓ HF checksums availableupdated today
No torrent yet →

Qwen3.8-9B-Distill-Uncensored-Heretic-MTPLX-Q6G64

MTPLX / Apple Silicon optimized Q6G64 conversion of:

petruhonk/Qwen3.8-9B-Distill-uncensored-heretic

Repository:

xy10086/Qwen3.8-9B-Distill-Uncensored-Heretic-MTPLX-Q6G64

Highlights

  • Qwen3.8 9B multimodal architecture
  • Q6 affine quantization, group size 64
  • Native BF16 MTP head preserved
  • 15/15 MTP tensors
  • BF16 vision tower preserved
  • 333 vision tensors
  • Image input verified
  • MTP speculative decoding verified
  • Recommended MTP depth: 2
  • Tested on Apple M5 Pro, 48 GB unified memory

Provenance

Source model:

petruhonk/Qwen3.8-9B-Distill-uncensored-heretic

Upstream model:

empero-ai/Qwen3.8-9B-Distill

The source repository did not include the vision preprocessing sidecars required by MTPLX. Matching preprocessor_config.json and video_preprocessor_config.json were restored from the upstream empero-ai/Qwen3.8-9B-Distill repository.

The source vision weights were preserved by MTPLX Forge and stored in model-vision.safetensors.

Quantization

Main language-model trunk:

  • 6-bit
  • Group size 64
  • Affine quantization

MTP:

  • BF16
  • 15 tensors
  • Not requantized

Vision:

  • BF16
  • 333 tensors
  • About 870 MiB on disk

Forge recipe: forge-recipe.json

Tested software

  • MTPLX 2.11.2
  • MLX 0.32.2
  • mlx-lm 0.31.3

Recommended configuration

  • Profile: sustained
  • MTP depth: 2
  • Context window: 32768
  • Max response tokens: 8192

Benchmarks

MTPLX Forge verification

Mode Decode tok/s
AR 32.42
D1 58.91
D2 71.70
D3 70.90

Real Chinese chat

Depth Avg decode tok/s Avg TTFT
D1 49.41 0.159 s
D2 50.73 0.134 s
D3 46.38 0.135 s

Vision test

  • 869 prompt tokens
  • 336 completion tokens
  • 65.30 decode tok/s
  • Successful screenshot understanding and OCR
  • Approx. 10.53 GB peak runtime memory

See benchmarks.json for the machine-readable summary.

Usage

Pull:

mtplx pull xy10086/Qwen3.8-9B-Distill-Uncensored-Heretic-MTPLX-Q6G64

Serve:

mtplx serve \
  --model xy10086/Qwen3.8-9B-Distill-Uncensored-Heretic-MTPLX-Q6G64 \
  --host 127.0.0.1 \
  --port 8000 \
  --profile sustained \
  --generation-mode mtp \
  --depth 2 \
  --context-window 32768 \
  --max-tokens 8192 \
  --reasoning off \
  --reasoning-effort low \
  --no-auth \
  --yes

OpenAI-compatible endpoint:

http://127.0.0.1:8000/v1

Behavior

This model is derived from a low-refusal / censorship-ablated checkpoint. Lower refusal behavior does not imply higher factual accuracy.

Medical, legal, financial and other high-stakes outputs should be independently verified.

Credits

  • petruhonk/Qwen3.8-9B-Distill-uncensored-heretic
  • empero-ai/Qwen3.8-9B-Distill
  • MTPLX

License

Apache-2.0. Review the upstream repositories for their attribution and license requirements as well.