⚠️ DEPRECATED — Experimental Model

This model is an early experimental release from the kniv cascade research program and is no longer maintained. It predates the current 5-head cascade architecture (POS, NER, DEP, SRL, CLS) and the bottom-up layer-selective training methodology that produces our current production teacher.

Use the current production model instead: dragonscale-ai/kniv-deberta-nlp-base-en-large

The current model offers significantly better quality across all tasks, includes Semantic Role Labeling and Dialog Act Classification heads, and has reproducible benchmarks against standard public test sets.

This repository is preserved for reproducibility and historical reference. No further updates, bug fixes, or evaluation runs are planned.

---

kniv-deberta-v3-nlp-en

Multi-task NLP model for English that performs NER, POS tagging, dependency parsing, and sentence classification in a single forward pass. Designed for edge and embedded deployment as the core linguistic analysis layer of the uniko cognitive memory system.

This is the student model, trained via knowledge distillation from kniv-deberta-v3-large-nlp-en (435M parameter teacher). Shipped as an INT8-quantized ONNX model for efficient CPU inference.

Model Details

Why DeBERTa-v3-small?

Compared to DistilRoBERTa (82M parameters), DeBERTa-v3-small offers:

• Smaller transformer body (44M non-embedding params vs 82M) at the same depth (6 layers, 768 hidden); 142M total with 128K-token SentencePiece vocabulary
• Disentangled attention -- separate content and position representations improve token-level tasks like NER and dependency parsing
• RTD pretraining -- ELECTRA-style replaced token detection yields stronger contextual representations from fewer parameters

Results

Dev set (best epoch 10, 500-sample eval)

External benchmarks

The student matches or exceeds the teacher on NER and CLS external benchmarks despite being 3x smaller, likely due to regularization benefits of the smaller architecture and longer distillation training (10 epochs vs 5).

Architecture

Shared DeBERTa-v3-small encoder with four linear classification heads. One encoder forward pass produces all four outputs.

DeBERTa-v3-small encoder (shared, 142M params)
  +-- NER head: Linear(768, 37)     -- per-token BIO entity tags
  +-- POS head: Linear(768, 17)     -- per-token UPOS tags
  +-- Dep head: Linear(768, 1411)   -- per-token dep2label composite tags
  +-- CLS head: Linear(768, 9)      -- per-sequence dialog act (from [CLS] token)

Task heads

NER entity types (18)

The NER head uses the 18-type entity scheme from spaCy en_core_web_trf, encoded in BIO format (37 labels = 1 O + 18 x 2 B-/I-):

CLS dialog act labels (9)

dep2label encoding

Dependency parsing is reformulated as token classification using the rel-pos encoding from Strzyz et al. (2019). Each token receives a composite label encoding its head attachment, making dependency parsing compatible with the same multi-task token classification architecture used for NER and POS.

Each label has the format {signed_offset}@{relation}@{head_UPOS}:

+1@nsubj@VERB

This means:

• +1 -- the head is the 1st token of matching POS to the right
• nsubj -- the dependency relation to the head
• VERB -- the UPOS tag of the head token

Decoding back to a dependency tree is O(n) per sentence. The label vocabulary is constructed from the training data and typically contains 800--1200 unique composite tags for UD English EWT.

Knowledge distillation

The student model is trained using a combined loss:

L = alpha * L_hard + (1 - alpha) * T^2 * KL(softmax(z_s/T) || softmax(z_t/T))

Soft labels are generated per-task by running the trained teacher over the full training set. The student learns from both signals for all four task heads simultaneously.

Training data

GMB's 8 entity types are mapped to the 18-type scheme: per->PERSON, org->ORG, gpe->GPE, geo->LOC, tim->DATE, art->PRODUCT, eve->EVENT, nat->EVENT.

This model does not use CoNLL-2003 data.

Training configuration

Intended use

This model is designed for edge and embedded NLP as part of the uniko cognitive memory system. It provides the core linguistic analysis pipeline -- entity recognition, part-of-speech tagging, syntactic parsing, and sentence classification -- in a single efficient forward pass.

Primary use cases:

• On-device NLP for privacy-sensitive applications
• Real-time text analysis in resource-constrained environments
• Linguistic feature extraction for downstream cognitive memory operations

Out-of-scope uses:

• High-stakes decision-making without human review
• Languages other than English
• Documents exceeding 128 tokens without chunking

Usage with ONNX Runtime

Rust (ort crate)

use ort::{Session, Value};
use tokenizers::Tokenizer;

// Load model and tokenizer
let session = Session::builder()?
    .with_model_from_file("model-int8.onnx")?;
let tokenizer = Tokenizer::from_file("tokenizer.json")?;

// Tokenize
let encoding = tokenizer.encode("Caroline went to the hospital.", true)?;
let input_ids: Vec<i64> = encoding.get_ids().iter().map(|&x| x as i64).collect();
let attention_mask: Vec<i64> = encoding.get_attention_mask().iter().map(|&x| x as i64).collect();

// Run inference -- single forward pass, four outputs
let outputs = session.run(ort::inputs![input_ids, attention_mask]?)?;
let ner_logits = outputs["ner_logits"].extract_tensor::<f32>()?;
let pos_logits = outputs["pos_logits"].extract_tensor::<f32>()?;
let dep_logits = outputs["dep_logits"].extract_tensor::<f32>()?;
let cls_logits = outputs["cls_logits"].extract_tensor::<f32>()?;

// Argmax over last dimension to get predicted label indices

Python (onnxruntime)

import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

session = ort.InferenceSession("model-int8.onnx")
tokenizer = AutoTokenizer.from_pretrained("dragonscale-ai/kniv-deberta-v3-nlp-en")

encoding = tokenizer(
    "Caroline went to the hospital.",
    max_length=128, padding="max_length", truncation=True,
    return_tensors="np",
)

outputs = session.run(None, {
    "input_ids": encoding["input_ids"],
    "attention_mask": encoding["attention_mask"],
})

ner_logits, pos_logits, dep_logits, cls_logits = outputs
ner_preds = np.argmax(ner_logits, axis=-1)
pos_preds = np.argmax(pos_logits, axis=-1)
dep_preds = np.argmax(dep_logits, axis=-1)
cls_pred = np.argmax(cls_logits, axis=-1)

Important: Use This Model's Tokenizer

Always load the tokenizer from this repo, not from microsoft/deberta-v3-small. The upstream HuggingFace tokenizer may omit BOS/EOS special tokens, shifting all positions and producing incorrect results.

# Correct
tokenizer = AutoTokenizer.from_pretrained("dragonscale-ai/kniv-deberta-v3-nlp-en")

# WRONG — may omit special tokens
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-v3-small")

Limitations

• English only. No multilingual support.
• 128-token context window. Longer documents require sentence-level chunking before inference.
• NER coverage. Entity types are limited to the 18 spaCy categories. Domain-specific entities (e.g., biomedical, legal terminology) are not covered.
• CLS labels are GPT-classified. Dialog act labels are generated by GPT-5.4-nano, not human-annotated. Macro F1 reflects imbalanced rare labels (correction, filler).
• dep2label decoding errors. The rel-pos encoding can fail to reconstruct a valid tree if the predicted head POS does not exist in the expected direction. Such tokens receive a fallback head of -1.
• Quantization trade-off. INT8 quantization reduces model size and latency but may degrade accuracy by up to 0.5% on individual tasks.

Training and export

git clone https://github.com/rustic-ai/kniv-nlp-models
cd kniv-nlp-models

# 1. Train teacher
python models/deberta-v3-large-nlp-en/train.py

# 2. Generate soft labels from teacher
python models/deberta-v3-large-nlp-en/generate_soft_labels.py \
    --model-dir outputs/deberta-v3-large-nlp-en/best

# 3. Prepare student data
python models/deberta-v3-nlp-en/prepare_data.py

# 4. Distill student from teacher
python models/deberta-v3-nlp-en/distill.py \
    --soft-labels outputs/deberta-v3-large-nlp-en/soft_labels

# 5. Export to ONNX (INT8 quantized)
./scripts/export.sh deberta-v3-nlp-en

Citation

If you use this model, please cite the dep2label encoding:

@inproceedings{strzyz-etal-2019-viable,
    title = "Viable Dependency Parsing as Sequence Labeling",
    author = "Strzyz, Michalina and Vilares, David and G{\'o}mez-Rodr{\'\i}guez, Carlos",
    booktitle = "Proceedings of the 2019 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/N19-1077",
}

The DeBERTa-v3 base model:

@inproceedings{he2021debertav3,
    title = "{D}e{BERT}a{V}3: Improving {D}e{BERT}a using {ELECTRA}-Style Pre-Training with Gradient-Disentangled Embedding Sharing",
    author = "He, Pengcheng and Gao, Jianfeng and Chen, Weizhu",
    booktitle = "International Conference on Learning Representations",
    year = "2023",
    url = "https://openreview.net/forum?id=sE7-XhLxHA",
}

Knowledge distillation approach:

@article{hinton2015distilling,
    title = "Distilling the Knowledge in a Neural Network",
    author = "Hinton, Geoffrey and Vinyals, Oriol and Dean, Jeff",
    journal = "arXiv preprint arXiv:1503.02531",
    year = "2015",
}