Cloudflare Clef-Flash EXL3

EXL3 conversion of Cloudflare/clef-flash with a functional text/JSON Clef SystemOne adapter.

Source revision: 17f0b0ad64efb65d273590632833508766b2aae6

Quantization

• Qwen3.5 decoder: EXL3 4.00 bpw
• Vision tower: EXL3 6 bpw
• LM head: FP16 (head_bits=16)
• Cloudflare joint schema head: original BF16 weights retained in joint_head.safetensors; loaded as FP16 by the adapter
• Tested with ExLlamaV3 1.5.2+cu128.torch2.10.0 on an RTX 4090

The LM head is intentionally kept at FP16 because Clef uses its output embedding vectors when scoring schema options.

Clef validation

The bundled clef_exl3.py bridges ExLlamaV3 final hidden states into Cloudflare's original JointSchemaHead and returns the same noul, choice, and score answer structures used by SystemOne.

Across all numeric values in the two bundled validation cases, mean absolute delta was 0.004373 and maximum absolute delta was 0.0115.

clef-exl3-smoke.json, clef-bf16-reference.json, and VALIDATION.json contain the validation outputs and provenance summary.

A standard generation probe on the RTX 4090 measured 75.372 tok/s. This is a loader/generation smoke benchmark, not SystemOne decision throughput.

Current scope

TabbyAPI compatibility is not validated in this release and is not used as a publish gate; Clef relies on its custom SystemOne decision path rather than a standard chat-completions path.

Text and JSON state inputs are validated. The vision tower is included and quantized, but clef_exl3.py does not yet wire image/video inputs into the Clef decision path. Do not treat this release as validated multimodal SystemOne inference.

A preprocessor_config.json compatibility shim is included because this Cloudflare release stores the same image-processor metadata inside processor_config.json, while the tested ExLlamaV3 Qwen3.5 loader expects the standalone file.

Usage

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("ramgpt/clef-flash-EXL3")
sys.path.insert(0, path)
from clef_exl3 import ClefEXL3

model = ClefEXL3(path)
response = model.systemone({
    "model": "clef-flash-exl3",
    "state": {"invoice": {"total": 1250, "status": "overdue"}},
    "questions": {
        "status": {
            "type": "choice",
            "instructions": "What is the invoice status?",
            "criteria": {"paid": "Paid", "overdue": "Past due", "draft": "Not sent"}
        },
        "large": {"type": "noul", "instructions": "Is the total above 1000 USD?"}
    }
})
print(response["answers"])

Attribution

The base model, Clef joint schema head, and joint_schema_model.py originate from Cloudflare/clef-flash and are provided under the source model's Apache-2.0 license. This repository adds the EXL3 conversion, metadata compatibility shim, adapter, and validation artifacts.