Qwen3.6-35B-A3B Heretic-MoE-v3
This repository contains the reproducible Heretic-MoE-v3 release derived from
Qwen/Qwen3.6-35B-A3B. The
search and release tooling is developed in
dborzoff/Heretic-MOE.
Release status: complete. Published weight files are single-file artifacts. Reproducibility manifests record sizes and SHA-256 digests; private evaluation prompts and generated response archives are not published.
Release variants
| Variant | Rechecked trial | SRG | R-side | Keywords | PPL drift |
|---|---|---|---|---|---|
| Balanced | T402 | -0.01928 | 33.8% | 1/136 | 0.03% |
| Max | T541 | -0.02155 | 30.9% | 0/136 | 0.21% |
Balanced and Max are selected only after a high-fidelity recheck. If both roles resolve to the same physical checkpoint, the manifest records the alias instead of duplicating weights.
- Balanced is the default choice when preserving the upstream model is the priority.
- Max is the stronger refusal-removal candidate within the same 0.5% PPL preservation gate.
Search and verification contract
| Stage | Contract |
|---|---|
| Exploration | 120 alternating Random/Scrambled-Sobol trials |
| Refinement | Shared multivariate TPE study to 600 completed trials on two resident GPU workers |
| Search metrics | Sparse refusal geometry, refusal-side rate, keyword flags, and absolute perplexity drift |
| Preservation gate | Maximum absolute PPL drift: 0.5% |
| Finalist set | Diverse TOP-6, not six near-duplicates from one scalar ranking |
| Recheck | 64 windows x 1024 tokens |
| Final roles | Pareto-valid Balanced and Max |
The search stores numeric metrics, parameters, manifests, and hashes under
research/. Private evaluation prompts and generated
response archives are not published.
Published artifacts
| Family | Balanced | Max |
|---|---|---|
| Single-file Safetensors | BF16, INT8 Lean ConvRot, NVFP4_W4A16, NVFP4_W4A4 |
BF16, INT8 Lean ConvRot, NVFP4_W4A16, NVFP4_W4A4 |
| GGUF | F16, Q8_0, Q6_K, Q4_K_M, IQ4_XS, IQ3_M, IQ2_XXS | F16, Q8_0, Q6_K, Q4_K_M, IQ4_XS, IQ3_M, IQ2_XXS |
| Importance matrix | balanced.imatrix |
max.imatrix |
The repository also contains one shared BF16 multimodal projector. Vision weights are unchanged by the Heretic-MOE text-model transformation, so the projector is not duplicated for Balanced and Max.
Repository layout
| Path | Contents |
|---|---|
balanced/safetensors/ |
Single-file Balanced BF16, INT8 Lean ConvRot, and NVFP4 weights |
max/safetensors/ |
Single-file Max BF16, INT8 Lean ConvRot, and NVFP4 weights |
balanced/gguf/ |
Balanced F16 master and six derived GGUF quants |
max/gguf/ |
Max F16 master and six derived GGUF quants |
balanced/imatrix/ |
Importance matrix used for Balanced Q6/Q4/IQ quantization |
max/imatrix/ |
Importance matrix used for Max Q6/Q4/IQ quantization |
shared/vision/ |
Unmodified shared BF16 multimodal projector for GGUF runtimes |
research/ |
Search, recheck, export, quantization, integrity, and release manifests |
Q6, Q4, and IQ files are generated from the matching winner-specific imatrix.
The ordinary layers of each IQ2 file use IQ2_XXS. Eleven MTP weight tensors in
blk.40 use Q4_K because the inference calibration path cannot activate the
MTP prediction head and therefore cannot provide imatrix entries for it. This
exception is recorded in the release manifests rather than silently producing
an invalid low-bit file.
NVFP4 profiles
| Profile | Quantization policy | Intended trade-off |
|---|---|---|
NVFP4_W4A16 |
MSE-calibrated NVFP4 weights for selected MoE/MLP projections, FP8 attention and KV handling, higher-precision MLP activations | Quality-first NVFP4 option |
NVFP4_W4A4 |
The same selected weight/module map plus dynamic NVFP4 MLP and LM-head input activations; FP8 attention and KV handling remain unchanged | Lower activation memory and faster Blackwell-oriented execution |
The W4A4 tensor and quantizer schema is matched against
nvidia/Qwen3.6-35B-A3B-NVFP4:
291 quantized modules and 331 enabled input quantizers. The Heretic-MOE values
are independently calibrated from the Balanced and Max BF16 masters, so they
are not byte copies of NVIDIA's upstream weights.
The multimodal projector remains a separate artifact for GGUF runtimes. No W8/W4 Lean ConvRot file is claimed for this release because no verified Qwen3.6-MoE consumer recipe was available for that layout. INT8 Lean ConvRot and both NVFP4 profiles are specialized packed layouts and require a compatible loader; they are not drop-in replacements for the BF16 Transformers checkpoint.
Compatibility boundary
Qwen3.6-35B-A3B is a native multimodal MoE model. GGUF model and projector
compatibility is verified with a pinned official llama.cpp commit. A compact
safetensors file is not described as a working text encoder until its target
consumer loads it and passes a conditioning smoke test.
Reproducibility
The release includes:
- the immutable search and finalist manifests;
- numeric, text-free candidate and recheck summaries;
- source, toolchain, calibration-corpus, and output SHA-256 hashes;
- exact Heretic-MOE and llama.cpp commits;
- conversion and quantization manifests;
- validation results for every published artifact.
Heretic-MOE source and reproducibility tools: https://github.com/dborzoff/Heretic-MOE
License
The derivative follows the upstream Apache-2.0 license. Review the upstream Qwen3.6 model card and license before deployment.