INTRIQ-SmolLM2-135M-Instruct

INTRIQ-SmolLM2-135M-Instruct is an instruction-tuned, surgically weight-edited 135M parameter language model based on HuggingFaceTB/SmolLM2-135M-Instruct. It is engineered for structured multi-turn engineering workflows, robust instruction following, and mathematical/scientific Python code generation.

---

🔬 Architectural & Evaluation Highlights

• Base Architecture: 30 Transformer layers, $d_{\text{model}} = 576$, $d_{\text{intermediate}} = 1536$, 9 attention heads.
• Surgical Weight Editing: Parameter modification targeted 12 causal tensors across intermediate and late layers (layers 14, 16, 18, 20, 22, 24, 26, 28, 29) to eliminate conversational amnesia and reinforce scientific reasoning without matrix degradation.
• Inside-Out Spectral Health (WeightWatcher RMT):
  • Evaluated via Random Matrix Theory (RMT) Empirical Spectral Density (ESD) power-law fitting ($\rho(\lambda) \sim \lambda^{-\alpha}$).
  • Global Mean Base $\alpha$: 6.4079 $\to$ Calibrated $\alpha$: 6.4121 (Optimal heavy-tail regime: $2.0 \le \alpha \le 6.0$).
  • Zero Spectral Collapse: 0 / 212 layers collapsed ($\alpha \ge 4.11$ across all surgically edited matrices).
• Promptfoo Heuristic Assertions: 5 / 5 passed on specific structural keyword prompts.

---

📊 Empirical Academic Benchmarks & Evaluation Realities

1. Measured Academic Benchmarks (lm-eval v0.4.13 on CPU):

2. Real Codebase Comprehension & Ambiguity Limitations:

• Repository Architecture Comprehension: 42.17 / 100 (Tested on real scientific codebases new_planet_identification and Bio-Sync-AI). The single-layer weight update produced zero net delta on broad repository-level reasoning.
• Ambiguity & Clarification Stress Test: 0.0 / 8.0 (0%). When presented with ambiguous deployment requests without credentials, the 135M parameter capacity defaults to emitting code completions rather than recognizing missing information and asking for clarification.
• Hugging Face Open LLM Leaderboard: Officially queued (status: pending, commit df4040f0...).

---

🚀 Quickstart & Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BRR1154/INTRIQ-SmolLM2-135M-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

messages = [
    {"role": "user", "content": "For a biosignal telemetry pipeline, write a Python function with type hints to compute the rolling Z-score across a 1D NumPy array with window size W."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=180,
        do_sample=False,
        repetition_penalty=1.15
    )

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

📊 Benchmark Summary

1. Architectural & Behavioral Suites

2. Standard Open LLM Leaderboard Benchmarks (EleutherAI lm-eval v0.4.13)

---

📜 Citation & Credits

• Base Architecture: SmolLM2 by Hugging Face (HuggingFaceTB/SmolLM2-135M-Instruct)
• Optimization Methodology: INTRIQ Multi-Tensor Closed-Loop Surgery & Random Matrix Theory (RMT) Auditing.
• License: Apache 2.0