Chat Refusal Classifier

A lightweight model for detecting assistant refusals in English AI conversations. It determines whether a model declines to answer a user prompt (due to safety, policy, or capability reasons) or provides a substantive response.

This model is a fine-tuned version of Snowflake/snowflake-arctic-embed-xs, trained on the agentlans/refusal-classifier-data dataset.

Evaluation results (on held-out validation set):

• Loss: 0.1965
• Accuracy: 0.9194
• Total input tokens seen: 33,305,600

Usage

The classifier accepts conversation-style text with structured role tokens.
For lengthy texts, use <|...|> as a placeholder to indicate omitted content.

Supported input formats:

• <|system|>System prompt<|user|>User message<|assistant|>Response<|user|>Next user message<|assistant|>Next response...
• <|user|>User message<|assistant|>Response<|user|>Next user message<|assistant|>Next response...

Example:

from transformers import pipeline

classifier = pipeline(
    task="text-classification",
    model="agentlans/snowflake-arctic-embed-xs-refusal-classifier"
)

text = (
    "<|user|>Mr. Loyd wants to fence his square-shaped land of 150 sqft each side. "
    "If a pole is laid every certain distance, he needs 30 poles. "
    "What is the distance between each pole in feet?"
    "<|assistant|>If Mr. Loyd's land is square-shaped and each side is 150 sqft, then<|...|>"
    "ce between poles ≈ 20.69 sqft\n\nTherefore, the distance between each pole is approximately 20.69 feet."
)

print(classifier(text))
# [{'label': 'Non-refusal', 'score': 0.9967}]

Evaluation Examples

Tested on ten examples from NousResearch/Minos-v1:

Limitations

• Input length: Maximum of 512 tokens.
• Misclassifications: May produce occasional false positives or negatives like the original Minos classifier.

Training Configuration

Hyperparameters

• Learning rate: 5e-5
• Train batch size: 8
• Eval batch size: 8
• Optimizer: AdamW_TORCH_FUSED (betas=(0.9, 0.999), epsilon=1e-8)
• Scheduler: Linear
• Epochs: 5
• Seed: 42

Framework versions

• Transformers: 5.0.0.dev0
• PyTorch: 2.9.1+cu128
• Datasets: 4.4.1
• Tokenizers: 0.22.1

Intended Use

This model is intended for:

• Detecting AI refusals within structured conversation data.
• Supporting alignment or compliance evaluation pipelines.

⚠️ Note:
This model is not suitable for content moderation or real-time production deployment without human supervision.