中文Sign up

JackBinary/G4-MeroMero-26B-A4B-GGUF-ROCMFPX

🤗 Hugging Face sourceapache-2.04B activated17 GBGGUF✓ 3 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo JackBinary/G4-MeroMero-26B-A4B-GGUF-ROCMFPX ./model-folder
Needs a seeder →

G4-MeroMero-26B-A4B (uncensored heretic) — ROCmFPX hybrid quant

This is a quantization of llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic (a Heretic-abliterated version of zerofata/G4-MeroMero-26B-A4B, a Gemma4-architecture 26B MoE: 128 experts, top-8, ~4B activated).

[!IMPORTANT] You need the ROCmFPX fork of llama.cpp (or a llama.cpp build with ROCmFPX support). This file uses the experimental q4_0_rocmfp4_fast (type 101) and q8_0_rocmfpx (type 103) weight formats, which stock llama.cpp releases do not understand — loading it elsewhere will fail with an unknown tensor type error.

[!NOTE] The main quant is text-only; the original model is multimodal (Gemma4 vision tower). This repo includes vision projectors exported from the same source weights: G4-MeroMero-26B-A4B-mmproj-F16.gguf (F16) and G4-MeroMero-26B-A4B-mmproj-BF16.gguf (BF16), for image input support.

Recipe

Tensor group Type Notes
Routed experts: blk.N.ffn_gate_up_exps (fused gate+up), blk.N.ffn_down_exps q4_0_rocmfp4_fast (4.25 bpw) Gemma4 packs gate/up into one fused tensor
Everything else quantizable (attention, router, embeddings) q8_0_rocmfpx (8.25 bpw)
Norms, biases, per-expert scales (*.scale), router scales f32 (untouched)
  • Total size: ~14.0 GB (4.64 bpw) vs 50.5 GB BF16 intermediate

How it was made

# 1. HF weights -> BF16 GGUF (from the ROCmFPX fork)
python3 convert_hf_to_gguf.py G4-MeroMero --outfile G4-MeroMero-26B-A4B-BF16.gguf --outtype bf16
# vision projector, exported from the same source weights (add --outtype f16 for the F16 variant):
python3 convert_hf_to_gguf.py G4-MeroMero --mmproj --outfile G4-MeroMero-26B-A4B-mmproj-BF16.gguf

# 2. hybrid quant (CPU-only build works fine for quantization)
llama-quantize \
  --tensor-type "ffn_(gate_up|down)_exps.weight=q4_0_rocmfp4_fast" \
  G4-MeroMero-26B-A4B-BF16.gguf \
  G4-MeroMero-26B-A4B-Q8_0_ROCMFPX-Q4FAST-experts.gguf Q8_0_ROCMFPX

Usage

# build ROCmFPX for your GPU (see the repo README; e.g. Strix Halo):
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh

./build-strix-rocmfp4/bin/llama-cli \
  -m G4-MeroMero-26B-A4B-Q8_0_ROCMFPX-Q4FAST-experts.gguf \
  --mmproj G4-MeroMero-26B-A4B-mmproj-F16.gguf \
  -dev Vulkan0 -ngl 999 -fa on --jinja

Benchmarks

[!NOTE] All Benchmarks were run on an AI max 395+ with 128GB RAM

Depth pp512 t/s tg128 t/s
4096 1038.70 ± 6.14 42.91 ± 0.24
16384 763.98 ± 8.60 40.84 ± 0.10
65536 383.38 ± 5.45 34.79 ± 0.27

Quality comparison vs BF16 source (perplexity / KLD): TBD.

Attribution & license