DecisionFairy-4B-GGUF

GGUF builds of DecisionFairy-4B for llama.cpp: the Qwen3.5-4B backbone and the decision head in one file (GGUF architecture = clef, clef.decision.{block_count, routing_block_count, head_count}), so one llama-server process answers typed decisions on /v1/systemone.

It is a pure decision model — no text generation. serves /v1/systemone and nothing else; GET /v1/models reports "output_modalities": ["decisions"].

Files

Text input only: no mmproj is published here, so the images / files fields of /v1/systemone are not available in this repo.

Run

Needs a llama.cpp build with the clef architecture. Upstream ggml-org/llama.cpp master has it (src/models/clef.cpp, LLM_ARCH_CLEF); older builds abort with an unknown-architecture error.

# bundled launcher: <gguf> <port> <ngl>
./start_llama.sh Qwen3.5-4B-Decision-Q8_0.gguf 18095 99

# or directly
llama-server -m Qwen3.5-4B-Decision-Q8_0.gguf -ngl 99 -c 4096 --port 18095

clef evaluates the whole prompt in one batch, so the prompt of a request must fit in --ubatch-size (default 512). For longer states and many options raise it, e.g. --ubatch-size 2048 -c 8192.

curl -s http://127.0.0.1:18095/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "The production API returns 502 and p99 latency is 15s.",
    "questions": {
      "severity": {
        "type": "choice",
        "instructions": "Assign the incident tier",
        "criteria": {"P0": "total outage", "P1": "partial degradation", "P2": "minor"}
      },
      "page_oncall": {"type": "noul", "instructions": "Page the on-call engineer now?"},
      "urgency": {"type": "score", "instructions": "How urgent is it?",
                  "criteria": ["can wait", "today", "right now"]}
    }
  }'

Response (shape from the upstream System One spec; values illustrative):

{
  "answers": {
    "severity":    {"type": "choice", "choice": "P0", "confidence": 0.98,
                    "probabilities": {"P0": 0.98, "P1": 0.015, "P2": 0.005}},
    "page_oncall": {"type": "noul", "noul": 0.99},
    "urgency":     {"type": "score", "score": 2.1, "confidence": 0.91,
                    "legend": {"0": "can wait", "1": "today", "2": "right now"},
                    "probabilities": {"0": 0.01, "1": 0.08, "2": 0.91}}
  },
  "usage": {"input_tokens": 210, "output_tokens": 0}
}

Question types: noul (yes/no probability), choice (one option of up to 255), score (2–10 ordered levels). The fields are decided jointly in one forward pass.

Differences from the safetensors release

• Quantized backbone and head: Q8_0 keeps close to bf16, Q4_K_M trades a little accuracy for 2.55 GiB.
• Text only — the vision tower shipped in the safetensors repo has no mmproj here.
• Probabilities are scaled with the temperatures stored in the file; they are not guaranteed to be calibrated for your data. Threshold on individual option probabilities, not on confidence.
• Same 1024-token training budget as the safetensors model: keep state + questions short.

License

Apache-2.0, following the Qwen3.5-4B backbone.