Humanizer 12B — MLX oQ5, group size 64

Text-only MLX conversion of jialinyyzz/humanizer, an English and Chinese text rewriting model. The upstream model and training are by jialinyyzz; this repository supplies converted weights and validation records.

Conversion

• Source revision: 2e57b6528a4070c33f3e09ca24e3325a84272f68.
• Source weights SHA-256: f6e8c0d176f9340cc2af82efac085e4ba292781bb0783a08eef07540d6b95548.
• oMLX revision: 70f44cfd652d2013dc0f9776dca7518e32101b00.
• Recipe: quantize_oq_streaming(oq_level=5, group_size=64, dtype="bfloat16", text_only=True, auto_proxy_sensitivity=True, trust_remote_code=False, enhanced=False).
• Mixed precision oQ5: base 5-bit affine weights with protected higher-precision modules. This is not uniform 5-bit quantization.
• Quantized module counts by precision: {"5": 293, "6": 18, "8": 18}.
• Weight storage: 8,485,097,843 bytes (7.90 GiB); measured effective weight storage 5.701 bits per parameter, including packing metadata.
• Non-quantized residual tensors remain BF16. Vision/audio components are removed for this text rewriting artifact.
• Calibration used a temporary 3-bit proxy to fit live memory, with a 512 MiB shard flush threshold. Final oQ5 weights were produced from the original BF16 source, not by re-quantizing that proxy.
• Versions: oMLX 0.7.0.dev4, MLX 0.32.2, mlx-lm 0.32.0.

Use with MLX

Use an MLX runtime that supports Gemma 4 unified models; the tested mlx-lm version is listed above. Build the plain rewrite prompt from prompt_format.json:

import json
from huggingface_hub import hf_hub_download
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

repo = "groxaxo/humanizer-12B-MLX-oQ5-G64"
model, tokenizer = load(repo)
with open(hf_hub_download(repo, "prompt_format.json"), encoding="utf-8") as f:
    pf = json.load(f)
draft = "Paste the draft to rewrite here."
prompt = pf["instr"] + "\n\n" + draft.strip() + pf["sep"]
print(generate(model, tokenizer, prompt=prompt, max_tokens=2048,
               sampler=make_sampler(temp=1.0, top_p=0.95, top_k=0)))

The included chat template constructs the author's rewrite prompt from the last user message, with one explicit BOS token and no chat turn markers. Both plain text and text-block messages are supported. The tokenized result matches the plain prompt with the tokenizer's normal BOS handling. The prompt body's fingerprint for the draft X is cc51d66b4c593fbe, matching the author-provided format. System messages and prior turns are not incorporated into the rewrite prompt.

The bundled generation settings follow the author's temperature 1.0, top-p 0.95, top-k disabled, min-p disabled, and repetition penalty 1.0. Stop on EOS. Use the author's 8192-token combined input/output budget and split longer documents at paragraph boundaries.

Validation

• Strict MLX weight loading passed.
• English and Chinese functional smoke tests passed with the fixed names, numbers and dates in their fixtures retained.
• Local oMLX streaming text completion and chat completion tests passed and reproduced the direct MLX rewrite exactly on the deterministic fixture.
• These small deterministic tests used temperature 0. They are not a repeat of the author's full benchmark, and BF16 quality parity was not measured.

See quantization-report.json, validation-local.json, validation-http.json, calibration-sensitivity.json, template-validation.json, and artifact-sha256.json for the exact observations and checksums.

To reproduce the memory settings with the pinned converter, set omlx.oq._PROXY_QUANT_BITS = 3 and omlx.oq._MAX_SHARD_BYTES = 512 * 1024 * 1024 before calling quantize_oq_streaming with the recipe above. The proxy ranked all 48 text layers using 128 calibration samples of 256 tokens.

The upstream model can make factual changes; review a rewrite before using it. The original license and attribution notice are included as LICENSE and NOTICE.