Clover-35B-A3B-Preview GGUF

GGUF conversion of Diffuspace/Clover-35B-A3B-Preview. All credit for the model goes to Diffuspace; the Apache-2.0 license carries over unchanged.

[!NOTE] Supported by mainline llama.cpp since commit e4b6c8ca6 (PR #31306, merged 2026-11-02). Older llama.cpp builds, Ollama, LM Studio and apps on an older llama.cpp do not know the clover_moe architecture and will fail to load it. These files were converted before the merge and patched in place to the merged tensor convention; see How it was made.

Model summary

Clover is a 35B-parameter mixture-of-experts diffusion language model with ~3B active parameters per token, derived from Qwen3.6-35B-A3B. It uses a hybrid Gated DeltaNet / full-attention backbone with:

• 40 layers
• 256 routed experts, top-8 routing
• 1 shared expert per layer
• block-diffusion generation
• 128K context window (131,072 tokens)

The hybrid backbone pairs full attention with Gated DeltaNet (GDN) layers; additional learned GDN parameters support bidirectional attention within diffusion blocks while preserving a causal prefix.

See the launch blog and the official code repo for full architecture and decoding details.

Benchmarks

DiffuSpace reports state-of-the-art performance among open-source diffusion language models at release; see the launch blog for evaluation settings and comparisons. These results describe the original BF16 model, not these quantizations. Quantization and inference settings can affect quality and speed.

Available files

[!NOTE] All files were made with llama.cpp's own presets and no importance matrix.

Recommendation: Q4_K_M is the no-imatrix sweet spot, fitting comfortably in 24 GB VRAM. Q6_K is the higher-quality option for anyone with 32 GB+ VRAM. IQ4_XS is smaller than Q4_K_M and roughly Q4_K_M-class without an imatrix, improving further with one. Q3_K_M is smaller but visibly degrades quality. Q3_K_S is the smallest 3-bit k-quant and degrades further. Q8_0 is effectively lossless.

How it was made

Two-step pipeline:

1. convert_hf_to_gguf.py --remote streamed the BF16 safetensors from the Hub into a BF16 GGUF. No .safetensors copy was ever written to disk.
2. llama-quantize (built from the pre-merge branch at 4838ffb0) quantized the BF16 GGUF down to each target type. The BF16 intermediate can be deleted afterwards.

# 1. BF16 GGUF via --remote (no .safetensors on disk)
python convert_hf_to_gguf.py --remote Diffuspace/Clover-35B-A3B-Preview \
    --outtype bf16 \
    --outfile Diffuspace-Clover-35B-A3B-Preview-bf16.gguf

# 2. BF16 -> quantized with llama-quantize from the same build
./build/bin/llama-quantize \
    Diffuspace-Clover-35B-A3B-Preview-bf16.gguf \
    Diffuspace-Clover-35B-A3B-Preview-Q4_K_M.gguf \
    Q4_K_M

These are k-quants plus one IQ4_XS, all made without an imatrix. Quantization is deterministic for a fixed source revision and llama.cpp commit, so these files match other no-imatrix conversions of the same model in the quantized tensor payload (only ssm_a and the GGUF layout can differ).

When the PR was reworked for mainline, the converter changed ssm_a to -exp(A_log) and dropped the block-diffusion-only tensors. These files were patched in place to match: the 30 ssm_a tensors were rewritten to -exp(A_log), the 120 legacy tensor entries were removed from the GGUF tensor table, and clover_moe.expert_shared_count was added. The quantized weights are unchanged. The removed payload remains as unreferenced bytes, so the file sizes are unchanged.

llama.cpp support

Clover support is in mainline llama.cpp since commit e4b6c8ca6, the merge commit of PR #31306 (2026-11-02). Build that commit or any later master:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout e4b6c8ca6   # or any later master
cmake -B build -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-cli llama-quantize

Drop -DGGML_CUDA=ON (or swap it for your backend) if you do not have a CUDA GPU. Released binaries built before that commit cannot run this model.

Running it

Plain decoding (recommended)

Every file runs as an ordinary causal model, one token at a time. The model defaults are greedy decoding with presence penalty 0.9 and repetition penalty 1.05. Thinking is on by default.

# OpenAI-compatible server
./build/bin/llama-server \
    -m Diffuspace-Clover-35B-A3B-Preview-Q4_K_M.gguf \
    -ngl 99 -fa on -c 32768 --jinja \
    --presence-penalty 0.9 --repeat-penalty 1.05 \
    --host 127.0.0.1 --port 8080

# CLI
./build/bin/llama-cli \
    -m Diffuspace-Clover-35B-A3B-Preview-Q4_K_M.gguf \
    -ngl 99 -fa on -c 32768

To turn thinking off, pass --chat-template-kwargs '{"enable_thinking": false}' (requires --jinja, which is enabled by default in this build).

Note: DiffuSpace's length and end-of-thinking controls (dllm_length_penalty, dllm_think_end_bias) belong to the diffusion sampler and are not implemented in llama.cpp.

Self-speculative decoding: not part of the merged PR

The original PR also exposed Clover's block-diffusion decoding as self-speculative draft-self, with no separate draft model. During review the maintainers asked to split everything speculative into a separate PR because it is not part of model support, so merged llama.cpp is AR-only. That PR has not been opened yet.

There is no draft-self spec type for Clover in merged llama-server, and no --spec-accept-ratio or --spec-accept-delta control. Use plain decoding.

Notes

• Q8_0 is effectively lossless for this model; expect output identical to BF16 within rounding noise.
• Small numerical changes (micro-batch size, K/V cache dtype) can shift Clover's predictions more than in typical models, so compare quantized files against each other on one fixed setup rather than against absolute numbers.
• The --remote conversion path is bandwidth-bound: one HTTP Range request per tensor, no connection reuse, so it takes roughly 1 to 2 hours for a 70 GB model even on a fast link. For repeat runs, hf download the safetensors locally once and convert from the local directory.
• Build flags for ROCm and other backends are unchanged from mainline llama.cpp.