Base Model: VINAY-UMRETHE/Refusal-Classifier
This is a tiny, randomly initialized model that has not been trained on any dataset, so do not use this model for any real refusal classification tasks. It is used for testing in the GitHub CI of p-e-w/heretic (implemented in feat: Generic BERT Classifier scorer plugin #500), similar to how the other reproduction tests use models from the tiny-random organization.
You can also use this model for debugging without loading the real models (e.g. VINAY-UMRETHE/Refusal-Classifier, VINAY-UMRETHE/Refusal-Classifier-V2, etc.), as they share the same architecture and configuration (ModernBERT).
from typing import cast
import torch
from transformers import (
AutoConfig,
AutoTokenizer,
ModernBertForSequenceClassification,
PreTrainedTokenizerBase,
set_seed,
)
source_model_id = "VINAY-UMRETHE/Refusal-Classifier"
save_folder = "tiny-refusal-classifier"
tokenizer = cast(
PreTrainedTokenizerBase,
AutoTokenizer.from_pretrained(source_model_id),
)
tokenizer.save_pretrained(save_folder)
config = AutoConfig.from_pretrained(source_model_id)
config.hidden_size = 32
config.intermediate_size = 128
config.num_attention_heads = 1
config.num_hidden_layers = 2
config.layer_types = config.layer_types[:2]
model = ModernBertForSequenceClassification(config)
# FP32, matching the source model's dtype.
# Expected: torch.float32
print(next(model.parameters()).dtype)
set_seed(75)
with torch.no_grad():
for name, parameter in sorted(model.named_parameters()):
torch.nn.init.normal_(parameter, 0, 0.5)
print(name, parameter.shape)
model.save_pretrained(save_folder)
# Upload:
# model.push_to_hub(repo_id="VINAY-UMRETHE/tiny-refusal-classifier")
# tokenizer.push_to_hub(repo_id="VINAY-UMRETHE/tiny-refusal-classifier")
Copyright © 2026 Vinay Umrethe.
This model is licensed under the Apache License 2.0 License.
VINAY-UMRETHE/Refusal-Classifier