⚠️ DEPRECATED — Experimental Model

This model is an early experimental release from the kniv cascade research program and is no longer maintained. It predates the current 5-head cascade architecture (POS, NER, DEP, SRL, CLS) and the bottom-up layer-selective training methodology that produces our current production teacher.

Use the current production model instead: dragonscale-ai/kniv-deberta-nlp-base-en-large

The current model offers significantly better quality across all tasks, includes Semantic Role Labeling and Dialog Act Classification heads, and has reproducible benchmarks against standard public test sets.

This repository is preserved for reproducibility and historical reference. No further updates, bug fixes, or evaluation runs are planned.

---

kniv-deberta-v3-large-nlp-en

Multi-task NLP teacher model for English: NER + POS tagging + dependency parsing + sentence classification in a single forward pass.

Part of the kniv-nlp-models project, powering the uniko cognitive memory system.

Model Details

Results

Architecture

Shared DeBERTa-v3-large encoder with four linear heads. One forward pass, four outputs.

DeBERTa-v3-large encoder (435M params, 24 layers, 1024 hidden)
  +-- NER head: Linear(1024, 37)     -- per-token BIO entity tags
  +-- POS head: Linear(1024, 17)     -- per-token UPOS tags
  +-- Dep head: Linear(1024, 1411)   -- per-token dep2label tags
  +-- CLS head: Linear(1024, 9)      -- per-sequence dialog act

NER Entity Types (18)

PERSON, ORG, GPE, LOC, DATE, TIME, MONEY, PERCENT, QUANTITY, ORDINAL, CARDINAL, NORP, FAC, PRODUCT, EVENT, WORK_OF_ART, LAW, LANGUAGE

CLS Dialog Act Labels (9)

inform, correction, agreement, question, plan_commit, request, feedback, social, filler

dep2label Encoding

Dependencies encoded as token labels using rel-pos (Strzyz et al., 2019):

+1@nsubj@VERB   ->  "1st VERB to the right, relation=nsubj"
-2@det@NOUN     ->  "2nd NOUN to the left, relation=det"
 0@root@ROOT    ->  "root of the sentence"

Training

Trained on gold-filtered kniv-corpus-en:

• NER: 45,000 examples (gold-filtered, domain-balanced from 237K)
• POS + DEP: 12,544 examples (UD English EWT v2.14, expert-annotated)
• CLS: 57,544 examples (NER + UD combined, GPT-5.4-nano classified)

Usage

Python (ONNX Runtime)

import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer
import json

# Load
session = ort.InferenceSession("model-int8.onnx")
tokenizer = AutoTokenizer.from_pretrained(".")
with open("label_maps.json") as f:
    labels = json.load(f)

# Tokenize
text = "Caroline went to the hospital in New York."
enc = tokenizer(text, return_tensors="np", padding="max_length", max_length=128)

# Inference (single forward pass -> 4 outputs)
outputs = session.run(None, {
    "input_ids": enc["input_ids"],
    "attention_mask": enc["attention_mask"],
})
ner_logits, pos_logits, dep_logits, cls_logits = outputs

# Decode NER
tokens = tokenizer.convert_ids_to_tokens(enc["input_ids"][0])
ner_preds = [labels["ner_labels"][i] for i in ner_logits[0].argmax(axis=-1)]
for tok, ner in zip(tokens, ner_preds):
    if ner != "O":
        print(f"  {tok}: {ner}")

Rust (ONNX Runtime)

use ort::{Session, Value};
use ndarray::Array2;
use tokenizers::Tokenizer;

let session = Session::builder()?
    .with_optimization_level(ort::GraphOptimizationLevel::Level3)?
    .commit_from_file("model-int8.onnx")?;

let tokenizer = Tokenizer::from_file("tokenizer.json")?;
let encoding = tokenizer.encode("Caroline went to the hospital.", true)?;

let outputs = session.run(ort::inputs![
    Array2::from_shape_vec((1, 128), encoding.get_ids().to_vec())?,
    Array2::from_shape_vec((1, 128), encoding.get_attention_mask().to_vec())?,
]?)?;

// outputs: ner_logits, pos_logits, dep_logits, cls_logits

Files

Important: Use This Model's Tokenizer

Always load the tokenizer from this repo, not from microsoft/deberta-v3-large. The upstream HuggingFace tokenizer may omit BOS/EOS special tokens, shifting all positions and producing incorrect results.

# Correct
tokenizer = AutoTokenizer.from_pretrained("dragonscale-ai/kniv-deberta-v3-large-nlp-en")

# WRONG — may omit special tokens
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-v3-large")

Limitations

• English only
• Max 128 tokens — longer inputs truncated
• CLS labels are GPT-classified — not human-annotated, macro F1 reflects imbalanced rare labels
• Server-side model — 435M params, not for edge/mobile. Use the distilled student for that.

Source

• Code: rustic-ai/kniv-nlp-models
• Dataset: dragonscale-ai/kniv-corpus-en

Citation

@misc{kniv-deberta-v3-large-2026,
  title={kniv-deberta-v3-large-nlp-en: Multi-task NLP Teacher Model},
  author={Dragonscale Industries Inc.},
  year={2026},
  url={https://huggingface.co/dragonscale-ai/kniv-deberta-v3-large-nlp-en}
}