GLiClass v4 Edge

Instruction-following zero-shot classification in a 0.4B encoder, built for CPUs and edge devices.

knowledgator/gliclass-v4-edge is the 0.4B-parameter edge release of GLiClass v4. The labels you pass and an optional instruction tell it what to decide at inference time. It returns a score for every label in one forward pass and generates no text. It reaches a Decision Index public skill of 47.5 at a median latency of 11 ms per decision, ahead of every model up to 1B parameters on the public board (the best of them scores 36.9).

Usage

pip install gliclass -U

Or install from source in a Python 3.10+ environment:

git clone https://github.com/Knowledgator/GLiClass.git
cd GLiClass
pip install -e .

The following examples reuse pipeline:

import torch
from gliclass import GLiClassModel, ZeroShotClassificationPipeline

model_id = "knowledgator/gliclass-v4-edge"
device = "cuda:0" if torch.cuda.is_available() else "cpu"

model = GLiClassModel.from_pretrained(model_id)
pipeline = ZeroShotClassificationPipeline(
    model,
    model_id,  # loads the tokenizer
    classification_type="multi-label",
    max_length=2048,
    device=device,
)

For a local copy, replace the model ID with the checkpoint directory.

Multi-label classification

results = pipeline(
    "The battery lasts two days, but the screen scratches easily.",
    ["battery", "display", "durability", "price", "shipping"],
    threshold=0.5,
)[0]
for result in results:
    print(result["label"], round(result["score"], 3))

Each prediction contains label and score. Pass a list of texts and batch_size=8 for batched inference.

Scores are calibrated on held-out data; predictions also carry initial_score, the uncalibrated score.

Single-label decisions and scoring

Use classification_type="single-label" to pick one answer: scores are a softmax over the labels. To score an input, use the levels of a scale as labels and read their probabilities:

answer = pipeline(
    "Our refund window is 30 days. The customer bought the item 45 days ago.",
    ["refund approved", "refund denied", "escalate to a human"],
    prompt="Apply the refund policy to the customer's request.",
    classification_type="single-label",
)[0]
print(answer)

quality = pipeline(
    "Q: What causes tides? A: Mostly the Moon's gravity, with a smaller effect from the Sun.",
    ["1 - wrong", "2 - poor", "3 - partly correct", "4 - good", "5 - excellent"],
    prompt="Rate the factual quality of the answer.",
    classification_type="single-label",
    return_hierarchical=True,
)[0]
print(quality)  # every level with its probability

Instruction following

Use prompt for the instruction, policy, or rubric. Labels are interpreted in light of it:

policy = (
    "Flag a message only if it shares another person's contact details without consent. "
    "Public business contact pages are allowed."
)
results = pipeline(
    [
        "You can reach our support team at help@acme.com.",
        "Here is my neighbour's personal phone number: 555-0134, call her at night.",
    ],
    ["violates policy", "complies with policy"],
    prompt=policy,
    classification_type="single-label",
)

Few-shot examples (examples=[{"text": ..., "labels": [...]}]) and hierarchical labels ({"sentiment": ["positive", "negative"], "topic": ["product", "service"]}) are also supported.

Knowledge graph reasoning

Pass the graph as text next to the story, the question as the instruction, and the answers as labels:

graph = (
    "Alice -member_of-> Finance team\n"
    "Finance team -can_read-> Reports folder\n"
    "Reports folder -contains-> Salaries.xlsx\n"
    "Alice -explicit_deny-> Salaries.xlsx\n"
    "Rule: explicit restrictions override team permissions."
)
answer = pipeline(
    graph,
    ["yes", "no"],
    prompt="Can Alice open Salaries.xlsx?",
    classification_type="single-label",
)[0]
print(answer)  # [{'label': 'no', 'score': ...}]

The same pattern covers multi-hop questions, agent and web-browsing steps (which link, tool, or action comes next), and NLI (entailment, neutral, contradiction).

Reranking

query = "How do I reset a forgotten router password?"
passages = [
    "Hold the reset button for 10 seconds to restore factory settings, then log in with the default password.",
    "Our routers support Wi-Fi 7 and dual-band operation.",
    "To change your Wi-Fi name, open the admin panel and edit the SSID field.",
]
scores = pipeline(
    passages,
    [query],
    prompt="Does the passage answer the query?",
    threshold=0.0,
)
ranked = sorted(zip(passages, scores), key=lambda p: -p[1][0]["score"])

Serving a Jev-style decision API

Decision engines such as Jev take a state (text or JSON) and a set of typed questions, and return a probability for every option of every question, with no generated text. GLiClass answers the same requests directly: each question is one GLiClass pass in which the state is the text, the question's instructions are the prompt, and every option is a label. This is the setup used for the Decision Index results below.

Probabilities are a softmax over the option logits. A JSON state is rendered as key: value lines, with string lists as - item lines.

The server below exposes POST /v1/systemone and GET /v1/models, the request and response shape of TypeSafe's System One API, so clients written for it (for example the WorkflowEvals runner) can point at a local GLiClass model. It needs only gliclass, fastapi, and uvicorn:

# jev_api.py
import json
import os

import torch
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

from gliclass import GLiClassModel, ZeroShotClassificationPipeline

MODEL_ID = os.environ.get("MODEL", "knowledgator/gliclass-v4-edge")
MAX_LENGTH = int(os.environ.get("MAX_LENGTH", "16384"))
DEVICE = "cuda:0" if torch.cuda.is_available() else "cpu"
DTYPE = torch.bfloat16 if DEVICE != "cpu" else torch.float32

model = GLiClassModel.from_pretrained(MODEL_ID, torch_dtype=DTYPE).to(DEVICE).eval()
pipe = ZeroShotClassificationPipeline(model, MODEL_ID, max_length=10**9, classification_type="single-label",
                                      device=DEVICE, progress_bar=False).pipe
app = FastAPI(title="GLiClass System One API")


class Request(BaseModel):
    state: str | dict | list | None = None
    questions: dict[str, dict]


def as_text(value):
    return value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)


def render_state(state):
    if state in (None, "", {}, []):
        return ""
    if isinstance(state, str):
        return state
    if isinstance(state, list):
        return "\n".join(f"- {as_text(item)}" for item in state)
    lines = []
    for key, value in state.items():
        if isinstance(value, list) and all(isinstance(item, str) for item in value):
            lines += [f"{key}:"] + [f"- {item}" for item in value]
        else:
            lines.append(f"{key}: {as_text(value)}")
    return "\n".join(lines)


def options(question):
    kind, criteria = question.get("type"), question.get("criteria")
    if kind == "choice":
        if not criteria or len(criteria) < 2:
            raise HTTPException(422, "a choice question needs at least two criteria")
        return list(criteria), [as_text(v) if v is not None else k for k, v in criteria.items()]
    if kind == "noul":
        c = criteria or {}
        return ["true", "false"], [as_text(c.get("true") or "yes"), as_text(c.get("false") or "no")]
    if kind == "score":
        if not criteria or len(criteria) < 2:
            raise HTTPException(422, "a score question needs at least two levels")
        return [str(i) for i in range(len(criteria))], [as_text(c) for c in criteria]
    raise HTTPException(422, f"unsupported question type {kind!r}")


@torch.no_grad()
def probabilities(text, prompt, labels):
    inputs = pipe.prepare_inputs([text], labels, same_labels=True, prompt=prompt)
    length = inputs["input_ids"].shape[-1]
    if length > MAX_LENGTH:  # refuse instead of truncating
        raise HTTPException(422, f"input of {length} tokens exceeds the {MAX_LENGTH}-token limit")
    logits = model(**inputs, max_num_classes=len(labels)).logits[0, : len(labels)].float()
    return torch.softmax(logits, dim=-1).tolist(), length


@app.get("/v1/models")
def models():
    return {"models": [{"name": MODEL_ID, "description": "GLiClass"}]}


@app.post("/v1/systemone")
def systemone(request: Request):
    text, answers, tokens = render_state(request.state), {}, 0
    for name, question in request.questions.items():
        keys, labels = options(question)
        prompt = as_text(question.get("instructions") or "") or None
        probs, length = probabilities(text, prompt, labels)
        tokens += length
        p = dict(zip(keys, probs))
        best = max(keys, key=p.get)
        if question["type"] == "choice":
            answers[name] = {"type": "choice", "choice": best, "confidence": p[best], "probabilities": p}
        elif question["type"] == "noul":
            answers[name] = {"type": "noul", "noul": p["true"]}
        else:
            answers[name] = {"type": "score", "score": sum(int(k) * v for k, v in p.items()),
                             "confidence": p[best], "probabilities": p}
    return {"model": MODEL_ID, "answers": answers, "usage": {"input_tokens": tokens, "output_tokens": 0}}

pip install gliclass fastapi uvicorn
uvicorn jev_api:app --host 0.0.0.0 --port 8765

curl -s localhost:8765/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "Checkout errors hit 12% after the 14:05 deploy.",
            "facts": ["Rolling back takes two minutes.", "No incident is open yet."]},
  "questions": {
    "action": {"type": "choice", "instructions": "What should the on-call engineer do right now?",
               "criteria": {"rollback": "Roll back to the last healthy build.",
                            "wait": "Watch error rates for another hour.",
                            "escalate": "Page the payments team lead."}},
    "urgent": {"type": "noul", "instructions": "Does this need action within the next hour?",
               "criteria": {"true": "Yes, customers are affected right now.", "false": "No, it can wait."}},
    "severity": {"type": "score", "instructions": "Rate the incident severity.",
                 "criteria": ["minor", "moderate", "major", "critical"]}
  }}'

The response has one answer per question. This one comes from gliclass-v4-edge (rounded):

{
  "model": "knowledgator/gliclass-v4-edge",
  "answers": {
    "action": {"type": "choice", "choice": "rollback", "confidence": 0.938, "probabilities": {"rollback": 0.938, "wait": 0.059, "escalate": 0.003}},
    "urgent": {"type": "noul", "noul": 0.300},
    "severity": {"type": "score", "score": 0.24, "confidence": 0.832, "probabilities": {"0": 0.832, "1": 0.124, "2": 0.013, "3": 0.031}}
  },
  "usage": {"input_tokens": 172, "output_tokens": 0}
}

Here the edge model picks the rollback but rates the incident as minor and not urgent; check judgment-style questions such as severity on your own data before relying on them.

• Instructions matter. Put the decision rule or rubric in instructions and give noul questions explicit true / false descriptions; bare yes / no labels are less reliable.
• Throughput. The server answers questions one at a time. For batch jobs, group questions that share a state with pipe.prepare_inputs([text] * n, labels_per_question, same_labels=False, prompt=prompts) and run one forward pass.
• Long inputs over MAX_LENGTH tokens are refused with HTTP 422, never truncated, as the Decision Index rules require.
• Calibration. For calibrated checkpoints logits are the calibrated logits; uncalibrated_logits holds the raw ones.
• The Decision Index results below come from a kit engine (gliclass_engine.GLiClassEngine) that applies exactly this mapping, run with python -m decision_index run --engine gliclass_engine:GLiClassEngine --model knowledgator/gliclass-v4-edge.

Reported evaluation results

Columns Edge 0.4B, 9B, and 27B are gliclass-v4-edge, gliclass-v4-9b, and gliclass-v4-27b; this card is Edge 0.4B.

Decision Index 0.3

Public part of the Decision Index 0.3 (37 benchmarks in five areas, 140,178 requests), scored with the kit's own scorers. Skill is chance-corrected (0 = random guessing, 100 = perfect) and coverage-adjusted (unanswered = wrong); the public index weighs the areas 25.8 / 25.8 / 20.0 / 18.3 / 10.0%.

GLiClass models: one pass per question through the Jev-style mapping above, on one RTX PRO 6000 (this model: bf16, max_length=16384, batch size 8). Edge 0.4B, 9B, and 27B take a median of 11.0, 34.0, and 73.7 ms per request. Other models: the per-benchmark public results in the kit's board snapshot, with area scores recomputed using the kit's aggregation, which reproduces each published index exactly. The main board ranking (the Full score) also includes private tests; this table covers the public part only.

Every benchmark, as skill (★ = gold benchmark, weight 1.2 inside its area):

Per-benchmark Decision Index results

Knowledge & Reasoning

Language Understanding

Retrieval & Classification

Tools & Automation

Arts & Human Taste

Models up to 1B parameters

Edge 0.4B against the sub-1B entries of the live board (the jev-decision-index Space, data generated 2026-10-15): the eight highest-scoring ones, plus LiquidAI d1-omni-600M and GLiNER2.5-Decide. Sizes are the board's served parameters; models over 1B served parameters are excluded even when their name says 0.8B. Board rank is the position on the public index among the 168 board entries plus Edge 0.4B. Area scores are recomputed from each model's per-benchmark public results with the kit's aggregation and match every published index.

Per-benchmark Decision Index results, models up to 1B

Knowledge & Reasoning (models up to 1B)

Language Understanding (models up to 1B)

Retrieval & Classification (models up to 1B)

Tools & Automation (models up to 1B)

Arts & Human Taste (models up to 1B)

KGJevBench

Decision tasks from the GLiClass benchmark suite (KGJevBench): policy decisions, instruction following, knowledge-graph walking, tool calling and routing, judging, and typed Jev-style decisions. GLiClass models were run with the suite's run_sample_500.sh (up to 500 examples per task, bf16); the 12-relations row uses eval_kg_reasoning.py --mode stepwise --pad-relations 12. The other columns are the KGJevBench comparison results for those models on the same tasks and metrics; they were not re-run here. All values ×100; the best value in each row is bold. Averages are recomputed from the cells shown, so the reference columns can differ by up to 0.3 from averages computed on unrounded scores.

Several KGJevBench sets are written in the formats GLiClass is trained on (for example triple relevance, task routing, and tool calling), and tasksource-Jev overlaps the training data; read those rows with that in mind.

Training and intended use

Training had three stages:

1. Large-scale supervised training on a mixture that recasts classification, NLI, QA, reranking, extraction, tool calling, moderation, judging, and reasoning data as decisions over candidate labels, with instructions, label descriptions where available. Single-label, multi-label, and scoring examples are mixed in one run with a focal loss.
2. Reinforcement learning with a PPO-style objective whose actions are the label decisions. Rewards are task metrics (F1, accuracy, recall), with a value baseline, a clipped ratio, and optional KL and entropy terms.
3. Calibration: with the backbone frozen, a small calibrator reads the text and label representations and logit statistics and predicts a per-pair temperature and bias, fitted by negative log-likelihood on held-out decision data.

Use this model for zero-shot and instruction-following classification, scoring, reranking, natural language inference, knowledge graph and multi-step reasoning, guardrails, policy moderation, routing, and LLM-output judging. It returns label probabilities and does not generate text.

Limitations

• Label wording, instructions, domain, input length, and thresholds affect predictions.
• Knowledge-heavy and multi-step reasoning tasks are the main gap to the larger models: multi-step arithmetic (GSM8K), hard science and exam questions (GPQA, HLE, MMLU-Pro, MedMCQA), logic puzzles, and long agent-trace judging remain weak spots.
• Results are strongest in English; quality on other languages is lower.
• Calibrated scores are fitted on held-out data and can drift on domains far from training.
• Treat outputs used for safety, moderation, legal, or medical decisions as signals for review, not final verdicts.

Citation

@misc{stepanov2025gliclassgeneralistlightweightmodel,
      title={GLiClass: Generalist Lightweight Model for Sequence Classification Tasks}, 
      author={Ihor Stepanov and Mykhailo Shtopko and Dmytro Vodianytskyi and Oleksandr Lukashov and Alexander Yavorskyi and Mykyta Yaroshenko},
      year={2025},
      eprint={2508.07662},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2508.07662}, 
}

@misc{stepanov2026scxrouter,
  title  = {SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology},
  author = {Stepanov, Ihor and Smechov, Aleksandr and Shtopko, Mykhailo and Vodianytskyi, Dmytro and Lukashov, Oleksandr},
  year   = {2026},
  eprint = {2609.02292},
  archivePrefix = {arXiv}
}