Cloudflare-clef-IQ3_S-recovered-GGUF

These files are Cloudflare's Clef at IQ3_S for Xyntetik Runner. Clef-IQ3_S.clef.gguf is the full decision model (the IQ3_S backbone with Clef's own BF16 joint decision head, unchanged); Clef-backbone-IQ3_S.gguf is the text-only backbone; a quant-recovery LoRA adapter fits both. They are not a recovered quant by the house bar: the bar of mean KLD ≤ 0.05 and margin-qualified top-1 ≥ 97% is NOT met (0.0736 / 95.6% against the base's 0.0879 / 94.0%).

• Research artifact, not a general-purpose release.
• What was changed: nothing in the quantized backbone or in the decision head. A rank-16 LoRA adapter (F16, 234 MB) was trained by distillation from the BF16 parent's top-64 distribution, to pull the IQ3_S file's output back toward the parent. The head was added on 2026-10-28; the first upload carried the backbone only.
• Why it exists: a Clef file of about 12-13 GB for 16 GB cards that reads closer to the parent; no file of that size met the bar (the best, this IQ3_S base, read 0.0879 / 94.0%).
• Decisions, against the BF16 parent with the same head: 92.95% of 2,000 typed decisions get the parent's answer with the QR-001e adapter (92.25% without); none of the parent's 704 confident decisions (top probability above 0.80) changes with the adapter (1 without). The same figures hold against Cloudflare's own float32 code (checked 2026-10-29, evidence/fp32-reference.json). Descriptive; there is no bar for decisions.
• Parent model: Cloudflare/clef (a fine-tune of Qwen/Qwen3.8-27B).
• Useful for normal inference? For Clef's typed decisions, use Clef-IQ3_S.clef.gguf with the adapter: close to the parent (about 93% same answer), not equivalent; a public Q3_K_M of about the same size agrees slightly more often on decisions (93.55%) while reading much further from the parent as text. As a text model, the backbone is a slightly closer IQ3_S. Neither file meets the bar.
• Runner compatibility: Runner 1.6.1 or later, measured on the CPU route: decisions through --systemone or POST /v1/systemone, text through the usual routes, --lora on both files.

• Validation status (Runner support tiers): each file's row in the Runner roster says which tier it is in and the Runner version and date of its last validation run; nothing is re-tested on a schedule.
  • Clef-IQ3_S.clef.gguf: roster row
  • Clef-backbone-IQ3_S.gguf: roster row
  • clef-IQ3_S-recovery-QR-001e.lora.F16.gguf: roster row
  • clef-IQ3_S-recovery-QR-001b.lora.F16.gguf: roster row

Run it

Typed decisions, Clef's purpose: the full model with the adapter.

runner -m Clef-IQ3_S.clef.gguf --lora clef-IQ3_S-recovery-QR-001e.lora.F16.gguf --gpu off --systemone requests.jsonl

One request per line, in the body of Cloudflare's systemone() (state, questions and their options; see the parent card). Add "runner_logits": true to a request for the raw per-option logits. With --serve, post the same body to /v1/systemone. --gpu off is the measured route.

Text only, the backbone:

runner -m Clef-backbone-IQ3_S.gguf --lora clef-IQ3_S-recovery-QR-001e.lora.F16.gguf

Add --serve for the OpenAI-compatible server. This file is the Clef backbone in Runner's qwen35 view: the text model without the joint decision head. The adapter changes neither base file; without --lora each reads as the plain IQ3_S below.

Measured against the parent (the house claim)

Runner scripts/kld-compare-raw.py, teacher-forced word by word over /v1/completions with no chat template, the candidate against the BF16 parent (Clef-backbone-BF16.qwen35.gguf), both on the CPU, 500 scored positions of the held-out prose corpus (harness-prose-heldout.txt, Wikitext-2 test), stride 1, --ctx 4096, tie band 0.5 nats. The bar is mean KLD ≤ 0.05 AND margin-qualified top-1 ≥ 97.0% at ≥ 500 scored positions. No row meets it.

The base row reads the same on 2026-10-18 (Runner 1.3.1) and 2026-10-25 (the build of these readings), to the last digit. Failed positions are the first one or two of a run timing out on a cold server; each run still scored 500.

Decisions measured against the parent (added 2026-10-28; descriptive, no bar)

Runner --systemone with the native Clef architecture (backbone and joint head; the per-option logits softmaxed per question), CPU, 16 threads, --ctx 16384, on a clang build of Runner main 20070a02, the commit of the KLD rows. The eval set is LocalLLaMA/typed-decisions at 7189f46b, split all/test: 400 cases, 2,000 decisions. The reference is the BF16 parent as a native Clef GGUF with the same head. Input token ids and option lists equal the parent's in all 2,000 decisions of every row. The definitions were preregistered (QR-001e, stage A):

• same answer: the candidate's top option is the parent's;
• confident changed: the parent's top probability is above 0.80 (704 of the 2,000 decisions) and the candidate picks another option;
• accuracy: the top option equals the dataset's label;
• mean total variation: between the two per-question distributions.

"right → wrong / wrong → right" counts the decisions where the parent matched the label and the candidate did not, and the reverse. The QR-001b adapter was not measured on decisions.

Other public Clef files on the same instrument (2026-10-18, Runner main f1f0c538, each file scored natively as published, with its own quantized head). The IQ3_S row reads the same on both dates to the last digit (92.25%, 1, 73.45%, 0.0501), so the rows are comparable. The last column is the house claim's instrument from the source gate.

For decisions alone, turintech's Q3_K_M, at about the same size, agrees with the parent more often than this repo's files (93.55% against 92.95%), while its text distribution is much further from the parent (KLD 0.1845 against 0.0736).

Re-run on the published Runner 1.6.1 asset (release commit c8459f90582ebcc587b7a06f41ec04a15357d436, runner-linux-x86_64 sha256 23efbbb2796f5d120caf6c78ca80cdad655c2d8a5263783423f94917d65cb4b6): the first 10 of the 400 cases (50 decisions), the published Clef-IQ3_S.clef.gguf with the QR-001e adapter, CPU. 50 of 50 answers are identical to the recorded row, with a max |Δ logit| of 0.00032 (evidence/native-spotcheck-runner-1.6.1.json).

The decision head: what we did (added 2026-10-28)

The first upload (2026-10-27) carried only the backbone. Clef's decisions come from its joint schema head, a small transformer that reads the backbone's final hidden states and scores every option of every question. Without it these files could not produce Clef's typed outputs. This update adds the head and reports how the IQ3_S file's decisions compare with the parent's. Nothing was trained for it.

1. The head is Cloudflare's, unchanged. llama.cpp's Clef converter (conversion/clef.py, unmodified, llama.cpp 2f729cfa) wrote the parent's native GGUF Clef-BF16.clef.gguf from Cloudflare/clef at the revision below, joint_head.safetensors included (sha256 a010ac04f078e699988e4049cbea5e62c962393f59fec366640b64e8d69a4953). It was checked value by value (evidence/clef-head-check.json). 159 of the 160 head tensors hold exactly Cloudflare's BF16 values: the converter splits each of the 20 fused attention in-projections into q, k and v, and stores 100 small tensors (biases and norms) as F32 holding the BF16 values exactly. The 160th, decision.scales, holds the head's three learned scalars as the values Cloudflare's code uses at inference (the two logit scales exponentiated and capped at 100, the residual gate through a sigmoid), equal to float32 rounding (2.3e-8 relative).
2. Backbone and head in one file. A repack script built Clef-IQ3_S.clef.gguf on 2026-10-17 (evidence/clef-repack.json). Every backbone tensor is byte for byte from the IQ3_S base measured above (its digest re-checked), and every head tensor byte for byte from the parent conversion. The metadata keys are renamed from qwen35.* to clef.*, and the head's own keys are added. This is the file the decision rows measured (sha256 0abe5b70f7002c8a14f13f2472b3a39a457fe47bd3c48814f74aca8c0852372b).
3. Public copy. As for the backbone file, the two local-path metadata strings are replaced by file names and the header is re-padded. The tensor data is byte-identical to the measured file (evidence/PUBLIC-COPY-CHANGES.md).
4. The adapter is the same file for both bases. It changes 496 backbone projections and reaches the head only through the backbone's hidden states.
5. Measured: the parent, the IQ3_S file, and the IQ3_S file with each of two adapters were scored natively, head included, on 2,000 decisions (the section above). Then the published Runner 1.6.1 asset re-ran 10 of those cases.

Limits, read before quoting

• The bar is not met. The best adapter closes about 16% of the base's mean KLD and 1.6 points of the margin-qualified top-1; it does not make the IQ3_S file parent-equivalent.
• About 7% of decisions differ from the parent, with or without the adapter. The adapter raises agreement by 0.7 points and leaves no confident decision changed; it does not make the file decision-equivalent.
• Parent-agreement is not a capability benchmark. "No quality loss" is not claimed and is not supported by this evidence. The accuracy column uses the dataset's labels, which a teacher model generated. Its small rise is not evidence of better decisions: about as many decisions moved each way (58 against 75 with the adapter).
• Decision agreement is one eval set of 400 cases with five questions each, so the 2,000 decisions are not independent. It has no preregistered bar.
• The decision reference is Runner's native BF16 conversion. It matches Cloudflare's own float32 code. On a tiny Clef (a Qwen3.5-0.8B backbone with a random head): input ids identical in 10 of 10 cases, max |Δ logit| 1.9e-4, 100% same answer. On the 27B (checked 2026-10-29), Cloudflare's float32 transformers path, streamed block by block on the CPU, gives the native parent's answer on all 2,000 decisions: input ids identical in 400 of 400 cases, max |Δ logit| 0.011. Against it, every decision figure on this card is unchanged: 92.95% with QR-001e, 92.25% without, and no confident decision changed with QR-001e (evidence/fp32-reference.json). The float32 reference counts 703 confident decisions; the native parent counts 704. Correction: an earlier version of this card said this check needed more memory than the machine had. That was wrong. The streamed path needs about 20 GB; the check had simply not been run.
• The decision rows ran on a source build (20070a02); the published 1.6.1 asset was re-checked on 10 cases only. The CUDA route for this file's decisions is not measured.
• KLD is computed over the union of both sides' top-N tokens and renormalised, a stated approximation of the instrument.
• One prose corpus, 500 positions. The adapter was trained on half prose and half chat (Wikitext-103 and SmolTalk), with every passage that overlaps the claim corpus excluded, so the claim corpus is the same kind of text as half the training data, not the same text.
• The QR-001e row was scored with a variant of the script that sends two discarded warm-up requests before the scored positions (formulas unchanged); the other rows used the script unchanged.
• Runner-only and measured on the CPU route. Fit on a 16 GB card is the design target, not a measurement. The decision head is unquantized BF16 (256 MB): the full model is 12.68 GB, 12.91 GB with the adapter; the backbone and the adapter are 12.65 GB together.
• The end-to-end check (Runner --lora against the training pipeline's own forward) was run on the round-1 adapter (QR-001b: max |Δ log-probability| 0.0020 over 32 positions, bar 0.01). The later adapters were trained by Runner itself, so its served path is the trained path.

What was trained

Four rounds, each preregistered before its first step:

• QR-001b: a rank-16 / alpha-32 LoRA on every linear module of all 64 layers (DeltaNet and attention projections, MLP), trained in a torch pipeline for 500 steps by KL to the parent's top-64 plus tail, lr 2e-4 cosine.
• QR-001c: continued in Runner's own trainer at lr 2e-4; its kill rule stopped it at step 100 (the development loss rose with the restarted learning rate).
• QR-001d: the corrected continuation, lr 2e-5 constant from QR-001b's adapter, 100 steps.
• QR-001e: QR-001d continued with its optimizer state to step 200; the gain on the development sets was 0.77% against the 2% its protocol required, recorded as INSUFFICIENT-GAIN. Its terminal adapter is this repo's best file.

Files

Provenance and records

• Parent: Cloudflare/clef at revision 0b331204bb13fbd2ca93a64df1956f5b55478ce5, the revision the parent GGUFs, the base and the head were made from (the repo has since moved on; both revisions are Apache-2.0). BF16 parent GGUF sha256 82ccfef016eca2fdc40d3a44b845a0e046dbf6f8891bb11585a1cae95797ccfe (the qwen35 view, the claim's reference); BF16 parent native Clef GGUF sha256 4adfd24b76d4d6ee7804885c46a0c07aeea21cb29398815634f50f61281c2529 (the decision rows' reference).
• Base: the parent's backbone quantized to IQ3_S with llama.cpp llama-quantize and an importance matrix. The published base file (sha256 dafa8752f756f21c941ffee1df6ab93fec499eaeddfea7e5ba5ba601abf747cb) differs from the measured file (sha256 beb746dbf1aae13921081743f412b5f124d3d827a82be5483858d3a8677bc59f) only in the two metadata strings quantize.imatrix.file and quantize.imatrix.dataset, which hold file names here instead of local paths; its tensor data is byte-identical (data section sha256 adcd8260aa4556f80fb3c661d3f1022e4a557dc687d178747cfa4b51f526455f).
• Full model: the published Clef-IQ3_S.clef.gguf (sha256 1e142e60b7067673971bbcbcb7af316409f530a24857770d30dae492932f0efe) differs from the measured file (sha256 0abe5b70f7002c8a14f13f2472b3a39a457fe47bd3c48814f74aca8c0852372b) in the same two strings only; its tensor data is byte-identical (data section sha256 fb9a8a3bdbd2aa820debc8e8cd48e02a15686686f28d06349e49ec8c166d58ac).
• Instrument: Runner scripts/kld-compare-raw.py (sha256 81745b8e...; the QR-001e row's warm-up variant 4cda4f70...), on a clang build of Runner main 20070a02; the decision rows used the same build's --systemone (binary sha256 4dcbea703755b7f4988784830739e630ea89b95cd8d4b409f898df8acd80062c).
• Training records in the private research repository (commits: QR-001b 4eac3512c on its quant-recovery branch, QR-001c b5cf7da5e with Amendment 1 2d5b631fe, QR-001d 44615c94e, QR-001e 1b6fc3a89 and 166292f81, its report 829b42d24; the decision rows 5e3c8a4c8, b9fba4755, a5fb3d5f5 and abca09c34): the preregistrations, amendments and reports.

Licence

The adapter: Apache-2.0. The base and the full model are a quantization of Cloudflare/clef, which is released under Apache-2.0 (its own base, Qwen/Qwen3.8-27B, carries its upstream terms); the decision head is Cloudflare's, unchanged. These files are distributed under those terms.