FrontiersMind/Lumma-fev-4B-v2-GGUF

This is the GGUF version of FrontiersMind/Lumma-fev-4b-v2, for llama.cpp. All quantizations are included, from Q2_K to BF16.

Lumma-Fev-4B-v2 is a decision model. You give it a state (text) and typed questions; it returns a probability for every option in one forward pass. It never generates text, so there is nothing to parse. It works in English, Hindi, Tamil and Kannada. It is a full-weight fine-tune of Qwen3.5-4B-Base.

Use it

llama-server -m Lumma-fev-4B-v2-Q4_K_M.gguf -ngl 99 -c 8192 -b 8192 -ub 8192 --port 8080

Set -b and -ub to at least the length of your longest prompt (the model reads the whole prompt in one batch). Then call the /v1/systemone endpoint:

curl localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Customer writes: I was charged twice for my order last week.",
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": {"billing": "payments and invoices", "support": "product help", "sales": "new purchases"}}
  }
}'

For the Python package and more detail, see the original model.