🧬 Tyranid-BERT (164M INT8)

"The Hive Mind of Code Verification. It doesn't converse. It doesn't negotiate. It devours corrupted ASTs and purges regressions in 20 milliseconds."

---

⚙️ The Doctrine: Synaptic Code Verification

Generative code review agents waste seconds buffering chat tokens, offering apologies, and hallucinating justifications for broken patches.

Tyranid-BERT rejects generative babble.

Acting as the neural synapse of the Code Oracle neuro-symbolic engine, Tyranid-BERT takes the Linearized Subgraph DSL extracted from Tree-sitter ASTs and Tarjan Strongly Connected Components (SCC), and evaluates patch safety with ruthless, mathematically calibrated precision.

Where Qwen-Servitor acts as the brute cybernetic thrall devouring massive raw pull requests (262k tokens), Tyranid-BERT is the hyper-specialized synaptic assassin: a 164M bidirectional encoder quantized to INT8 (151 MB) executing multi-task code diagnosis in < 20 ms on commodity CPUs.

---

🔬 Synaptic Architecture & Multi-Task Heads

                   [ LINEARIZED AST SUBGRAPH DSL (< 400 TOKENS) ]
                                         │
                                         ▼
            ┌────────────────────────────────────────────────────────┐
            │             ModernBERT-base 164M Backbone              │
            │   • Bidirectional Unpadding FlashAttention             │
            │   • Rotary Position Embeddings (RoPE)                  │
            │   • 8,192 Token Native Receptive Field                 │
            │   • Zero Autoregressive Overhead (Single Forward Pass) │
            └────────────────────────────┬───────────────────────────┘
                                         │
                                [ [CLS] Latent Vector ]
                                         │
                 ┌───────────────────────┼───────────────────────┐
                 ▼                       ▼                       ▼
        ┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
        │ 1. RISK HEAD    │     │ 2. TAXONOMY     │     │ 3. UNCERTAINTY  │
        │ Continuous MSE  │     │ 5-Class BCE     │     │ Heteroscedastic │
        │ Calibrated [0-1]│     │ Multi-Label     │     │ Log-Variance    │
        │ T = 1.5967      │     │ Defect Vectors  │     │ Confidence Gate │
        └─────────────────┘     └─────────────────┘     └─────────────────┘

1. Multi-Task Output Telemetry

In a single forward inference pass (< 20 ms), Tyranid-BERT outputs:

---

⚡ Runtime Formats & Weights

This repository provides three production-grade model variants:

1. model_int8.onnx (151 MB — Recommended):
   • Dynamic INT8 quantized ONNX graph (targeting Linear/MatMul layers).
   • Zero-PyTorch footprint: runs with standalone onnxruntime (~50 MB wheel).
   • Sub-20ms inference on modern CPU architectures.
2. model.onnx (598 MB):
   • Full FP32 ONNX graph with dynamic axes (batch_size, sequence_length).
3. model.safetensors (598 MB):
   • Pure PyTorch weights for fine-tuning or Python research workflows.

---

🛠️ Quickstart

1. Installation

pip install onnxruntime transformers

2. Fast ONNX Runtime Inference (< 20ms)

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

MODEL_ID = "wxsys/tyranid-bert"

# Load tokenizer and INT8 ONNX session
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
session = ort.InferenceSession("model_int8.onnx", providers=["CPUExecutionProvider"])

subgraph_dsl = """[DIFF_TARGET] app/auth.py::verify_token (MODIFIED)
[METADATA] File: app/auth.py | OldLines: [10..15] | NewLines: [10..18]
[NODES]
N0: app/auth.py::verify_token [def verify_token(token: str)] (SEED, MODIFIED)
N1: app/routes.py::login_handler [def login_handler()]
[EDGES]
N1 -> N0 (CALLS)
[GATE]
STATUS: APPROVED (conf: 0.98)
CYCLES: 0
VIOLATIONS: NONE"""

# Tokenize (< 400 tokens)
inputs = tokenizer(subgraph_dsl, max_length=512, truncation=True, padding=True, return_tensors="np")
ort_inputs = {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64),
}

# Single forward pass
risk_score, taxonomy_logits, uncertainty = session.run(None, ort_inputs)

print(f"Risk Score: {float(risk_score[0]):.4f}")
print(f"Taxonomy Vector: {taxonomy_logits[0]}")

3. Integrated Use in Code Oracle CLI

Tyranid-BERT is the official neural verifier for Code Oracle:

# Verify a patch with sub-20ms INT8 neural decision head
code-oracle verify src/service.ts --patch patch.diff -w .

---

🧬 Training Dataset: Golden Hybrid v3

Tyranid-BERT was fine-tuned on the Golden Hybrid v3 dataset (~4,900 samples):

• Real Git Reverts & Hotfixes: Mined from production repositories across Go, Python, TypeScript, and Rust.
• Surviving Mutants: Hard negatives that strictly passed Stage 1 (AST Syntax) and Stage 2 (Tarjan SCC Cycles).
• Balanced Stratification: Exact 50% PASS / 50% REJECT distribution with zero data leakage.

---

📜 Citation & License

• License: Apache 2.0
• Base Architecture: answerdotai/ModernBERT-base
• Author / Maintainer: Wahyu Zero (@wahyuzero / wxsys)
• Project: Code Oracle