Qwen3.8-9B Heretic Uncensored - 5-bit MLX
This repository contains 5-bit MLX weights for rohit267/Qwen3.8-9B-heretic-uncensored.
The requested reference model was saga404/Qwen3.8-9B-heretic-uncensored-Q5_0-GGUF. MLX LM cannot convert GGUF weights directly. This conversion used the original BF16 safetensors weights to avoid an extra dequantization and requantization step.
Quantization
| Setting | Value |
|---|---|
| Weight bits | 5 |
| Group size | 32 |
| Mode | affine |
| MLX LM version | 0.31.3 |
| MLX version | 0.32.1 |
The final model uses about 6.001 bits per weight after scales, biases, and unquantized parameters are included.
Use
Install MLX LM:
pip install mlx-lm
Run the downloaded model:
mlx_lm.generate \
--model ./Qwen3.8-9B-heretic-uncensored-5bit-MLX \
--prompt "What is 2 + 2?" \
--max-tokens 128
Use the source model's recommended sampling settings for longer responses:
temperature=0.6
top_p=0.95
top_k=20
Validation
The converted weights loaded and generated a correct response to 2 + 2 on Apple silicon.
Peak memory during this smoke test was 6.859 GB.
License
The source model is licensed under Apache 2.0.