中文Sign up

JackBinary/ROCmFPX-Quantize-Playbook

🤗 Hugging Face sourcemit-GGUF⚠ No SHA-256 weightsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo JackBinary/ROCmFPX-Quantize-Playbook ./model-folder
Needs a seeder →

ROCmFPX Quantize Playbook

A field-tested, agent-ready playbook for producing ROCmFPX hybrid GGUF quantizations (q4_0_rocmfp4_fast / q8_0_rocmfpx) with the ROCmFPX fork of llama.cpp — fully CPU-only, from either pre-quantized GGUF repos (e.g. Unsloth BF16) or raw HF safetensors.

The complete guide is in rocmfpx-quantization-guide.md. Hand this repo (or just that file) to a coding agent together with a Hugging Face link and a recipe, and it can reproduce every step below.

What it covers

  • One-time CPU-only build of llama-quantize / llama-gguf-split + Python conversion deps
  • Workflow A: source repo already has BF16 GGUF shards (e.g. unsloth/*-GGUF)
  • Workflow B: raw safetensors → convert_hf_to_gguf.py → BF16 GGUF
  • How to discover routed-expert tensor names per architecture (regex lookup table)
  • Dry-run verification, shard merging, and output validation
  • VRAM-fit hybrid recipe (q4 bulk + q8 sensitive) derived from Unsloth Dynamic tiers
  • Cheatsheets, timings, and gotchas (regex anchoring, MTP auto-protection, gguf-py version trap…)

Validated runs

Model Recipe Result
Laguna-S-2.1 118B-A10B q4 experts / q8 rest 224 GB → 61.6 GB (4.39 bpw)
Qwen3.8-27B pure q8 and 16 GB hybrid from UD-Q4_K_XL tiers 26.9 GB (8.25 bpw) / 16.4 GB (5.15 bpw)
G4-MeroMero-26B-A4B q4 experts / q8 rest + mmproj 50.5 GB → 14.0 GB (4.64 bpw)

Timings on a 64-core CPU box: 118B MoE ≈ 8 min, 27B dense ≈ 1 min, 26B MoE ≈ 3 min.

Related