Clef-Flash — GGUF quants (language tower)

Cloudflare/clef-flash quantized for llama.cpp. The source is a 9.4B-parameter model (architecture Qwen3_5ForConditionalGeneration, 32 layers, hidden size 4096, vocab 248,320), post-trained by Cloudflare from Qwen/Qwen3.5-9B and released under Apache-2.0. This repo has 30 verified files: 29 GGUF quants plus the full-precision BF16 source.

⚠️ What these files do and don't contain

Clef-Flash is a decision model: it reads a state (text, JSON, images, or video) plus a schema of typed questions and returns a probability for every allowed option of every question in a single forward pass — no free-form text generation, no output parsing. That behavior comes from a backbone plus a separate joint schema head (and optionally the vision encoder).

This repo contains the language tower only.

• The vision encoder is not included — there is no --mmproj file here, so image and video inputs are not available from these GGUFs.
• The joint schema head is not included (joint_head.safetensors is a separate component in the source release). Per-question probabilities and the Jev / SystemOne API behavior therefore cannot be reproduced from these files.

For the full decision-model behavior, use Cloudflare/clef-flash directly with transformers (5.10.2) and torch (2.11) per its README. What you get from this repo in llama.cpp is the post-trained Qwen3.5-9B language tower.

About Clef-Flash

From the Cloudflare announcement: Clef-Flash turns a state plus a schema of typed questions into decisions. Each question is one of three types — noul (true/false), choice (named options), or score (ordered options) — and the model's joint schema head reads the backbone's final hidden states, routes evidence from the state to each question, and scores all options of all questions jointly. The Clef-Flash API is compatible with Jev and SystemOne. Clef-Flash is the smaller sibling of Cloudflare/clef.

Files

All 30 files below are verified present on the Hub.

Quantized 2026-10-04. The importance matrix was computed on the full BF16 weights, not on a lower-precision surrogate.

Usage

# run directly from the Hub
llama-cli -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M -p "Hello"

# or serve it
llama-server -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M

# or download a single file
huggingface-cli download NANI-Nithin/clef-flash-GGUF clef-flash-Q4_K_M.gguf --local-dir .
llama-cli -m clef-flash-Q4_K_M.gguf -p "Hello"

These files are llama.cpp GGUF. They are not loaded by vLLM, SGLang or transformers the way the source repo is — for that, use Cloudflare/clef-flash.

Which file?

• Q4_K_M (5.24 GB) — the default choice; best quality-per-byte for most setups.
• Q8_0 (8.87 GB) — effectively lossless, when resources allow.
• Q6_K (6.85 GB) — near-lossless middle ground.
• IQ4_XS (4.84 GB) — the best pick under ~4.5 bits per weight.
• IQ2_M (3.36 GB) — the smallest size most people find usable.
• Everything smaller than IQ2_M is experimental or extreme; expect substantial degradation. The lowest-bit files exist for completeness only.

Results

Cloudflare publishes decision-model benchmark results in the source README and on the Decision Index leaderboard. In Cloudflare's internal run of the Decision Index 0.2.1 suite, Clef-flash posts the best score among the compared decision models in the majority of listed rows — for example BFCL 98.8 (case exact accuracy), MMLU 91.8, HellaSwag 98.6, WinoGrande 97.5, and a median request latency of 38.8 ms — with ForecastBench reported as a Brier score (10.6), where lower is better. These numbers describe the source model with its joint head on Cloudflare's own harness; the quantized files in this repo were not re-evaluated.

Licence

Apache-2.0, following the base model Qwen/Qwen3.5-9B. Clef-Flash was created by Cloudflare; these quants are an independent llama.cpp conversion of the published weights.