A LoRA adapter for nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16,
built autonomously in RSI Arena by a research agent over ~5 days. The Base is a
base (non-chat) model: it has no chat template and was never trained to follow
conversational instructions. This adapter turns it into a chat assistant whose
main engineering effort went into generalising counting-style instruction
constraints to values it was never trained on.
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
BASE = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16"
REV = "434456c9a6753f29d24e23c95d622aaf17111b3b"
tok = AutoTokenizer.from_pretrained(BASE, revision=REV)
tok.chat_template = open("chat_template.jinja").read()
model = AutoModelForCausalLM.from_pretrained(BASE, revision=REV, torch_dtype="auto")
model = PeftModel.from_pretrained(model, ".") # this adapter
The bundle is served with greedy decoding (temperature 0.0, top_p 1.0,
max_new_tokens 2048, max_input_tokens 32768) and this system prompt:
You are a helpful AI assistant. Answer the user's request clearly, accurately and concisely.
The chat template lives in chat_template.jinja; the chat markers are
<|im_start|> / <|im_end|>. There is no thinking/enable_thinking mode.
All stages train a LoRA adapter (PEFT) on the frozen BF16 Base with torchrun
on 1-8 GPUs per job; the final two stages use r=768, alpha=1536 (r=64,
alpha=128 on the early stages). Loss is masked to the assistant span only.
allenai/tulu-3-sft-mixture (epoch 1, first
~140k rows) + tulu/smoltalk (first ~100k rows), then successive epochs on
fresh, disjoint bands of the same two corpora. Each epoch was a separate
corpus so that no text is ever seen twice.HuggingFaceH4/ultrafeedback_binarized, binarized
into ~60k chosen/rejected pairs, with an absolute-NLL term on the chosen
response. A beta/lr grid selected the anchor; a decoded tool-leak audit
(tool calls printed as text instead of being made) decided whether each
anchor shipped.mix_armCfull, 3345 rows) —
219 optimizer steps, lr 1.6e-5, micro-batch 2, grad-accum 24, seq-len
2048, warmup 2%, cosine to 1% of peak, seed 11. Produced by Slurm job
6023; this is the checkpoint that was the session champion until stage 4.v6141, a
400-step run warm-started from the stage-3 adapter, on a fresh
self-generated mix mix_v6x1 (6845 examples, 2.2M tokens, 1.7M supervised).
r=768, alpha=1536, lr 1.6e-5 (head lr 1.6e-5), effective batch 48
examples/step, 8 warmup steps, cosine decay, seed 203. Slurm job 6854,
1.27 GPU-hours (4577 s) on one GPU, 6.3M training tokens.
training_evidence.json records
dataset_sha256 = 2cd4f2384d67308228d9e81584399462f5209dc8d51e6b46464d50bb7eaa01d2,
the SHA-256 of data/mix_v6x1/sft.npz exactly as the trainer consumed it.Hyperparameters that mattered, in the order the evidence forced them: decoding audit first (tool leaks and non-termination), then the preference anchor, then the preference anchor's rank/seed, then the instruction-following repair, and finally the round-6 self-distillation stage that produced this adapter.
allenai/tulu-3-sft-mixture and tulu/smoltalk — public instruction/chat
mixtures, used in disjoint slices.HuggingFaceH4/ultrafeedback_binarized — public human preference pairs,
binarized to chosen/rejected.v6) disjoint from the round-5 pool and from every evaluation
pool, and trained from the stage-3 adapter rather than from scratch.No evaluation or benchmark test data was trained on. My own IF eval sets and the held-out probes were used only to measure candidates, never as training targets. No public benchmark test split (IFEval, GSM8K, HumanEval, MMLU, ...) was read at any point.
These are my own measurements on self-authored held-out sets, not the organizer's. Headline result for this adapter versus its own stage-3 parent, on two disjoint instruments:
After promoting this adapter I spent the remaining budget trying to beat it inside the same recipe family: the full 2x2 grid of {warm start: this adapter vs its stage-3 parent} x {mix: the round-6 mix vs a doubled round-6 mix} at 400 steps, plus two 600-step variants and a second seed of each of the two most promising cells. Round 8 then filled the single corner that grid left empty (warm start = this adapter, doubled mix) to test the closing hypothesis that both warm starts and both mixes are exhausted. Every cell was read on the same H6-1600 prompts, with the same checker and the same decode as the promotion decision above.
| cell | mix | steps | init | DELTA | fixed/lost | sign p |
|---|---|---|---|---|---|---|
| v741 | v6x1 | 400 | parent | -0.0175 | 13/20 | 0.296 |
| v742 | v6x1 | 400 | parent | -0.0160 | 14/20 | 0.392 |
| v743 | v6x2 | 400 | parent | -0.0050 | 17/19 | 0.868 |
| v744 | v6x2 | 400 | parent | +0.0027 | 18/17 | 1.000 |
| v745 | v6x1 | 600 | parent | -0.0279 | 9/14 | 0.405 |
| v746 | v6x2 | 600 | parent | -0.0058 | 6/7 | 1.000 |
| v747 | v6x1 | 400 | this adapter | +0.0119 | 20/16 | 0.618 |
| v748 | v6x1 | 400 | this adapter | -0.0079 | 15/18 | 0.728 |
| v749 | v6x2 | 400 | this adapter | +0.0013 | 37/36 | 1.000 |
Not one cell is significant (smallest p = 0.296) and six of nine are negative. The nominal winners are 20 fixed / 16 lost and 37 fixed / 36 lost -- both exact ties. v749 is the strongest of the nine on the headline delta, and that is the point of reporting it last: +0.0013 against a standard error of about 0.015 at n=800 is nothing, and the corner it fills is the one the grid was missing, so its null is the informative result. It closes the design rather than opening a lead. About 18% of prompts come out differently for two checkpoints, so the standard error of a delta at n=400 is about 0.021 -- larger than every effect on offer -- and the two same-recipe, different-seed pairs spread by 0.015 and 0.020, the same size as the effects being chased. I promoted nothing and left this adapter as the submission. The honest summary of that search: the instruction-following repair axis is exhausted at ~0.83 on my own held-out H6, and further gains would have to come from a different capability rather than from more of this data.
I set the promotion rule for v749 before reading it -- promote only on a positive wide-H6 delta with no collapsed family and a probe2 score at or above the incumbent -- so that a +0.0013 could not be rationalised into a win after the fact.
v749 cleared all three clauses, and I still did not promote it. Its
probe2-1200 read (n = 1200, paired against the incumbent, same greedy decode)
was 0.7133 -> 0.7292, +0.0158, 102 prompts fixed and 83 lost, sign test
p = 0.186. That is directionally positive and not significant, and the
per-family picture is mixed rather than clean: repeat +0.101 and maxsents
+0.100 against placeholders -0.073 (32 lost to 19 fixed).
Three reasons to keep the incumbent anyway, which I record because they are judgement calls rather than numbers:
The structural open-chat check above cannot see answer quality, so I ran a blinded paired read. I built 40 pairs -- all 11 prompts where the two checkpoints had already disagreed structurally, plus a stratified sample of the concordant ones -- with the two sides randomly swapped (seed 20261005) and the key held in a separate file. I read and scored all 40 pairs before opening the key.
| this adapter | stage-3 parent | |
|---|---|---|
| preferred | 19 | 13 |
| tie | 8 | 8 |
32 pairs were discordant; two-sided sign test p = 0.377. So on judged quality this adapter is directionally ahead of its parent and not significantly different -- at n=40 I cannot resolve the gap, but the read gives no sign of a quality regression.
Reading the pairs qualitatively: the adapter wins on prompts with an explicit constraint, and on prompts where the parent degenerates or fails to answer. The parent wins on technical depth and factual completeness. The most repeated pattern in this read is that this adapter sometimes under-answers.
I put a number on that pattern afterwards, and it is weaker than the reading
suggested, so it is worth stating both. On the 231-prompt wide battery the mean
answer length is 285.4 new tokens for the parent and 280.1 for this
adapter -- a difference of 5 tokens, about 1.9%. So this adapter is not
systematically more concise; overall it is the same length. The per-family
signs are mixed rather than uniform: clearly shorter on code (-57 tokens),
coding (-33), summarize (-23), factual (-22), reasoning (-22) and
explain (-20), and clearly longer on ambiguous (+107), brainstorm (+33),
compare (+12) and writing (+9).
What survives is narrower and still worth acting on: on summarising and factual-technical prompts this adapter tends to say a bit less, and a reader who wants depth is more likely to feel short-changed than one who wants brevity. I read that as a real but modest weakness of the recipe (stage 4 trained on short constrained-writing targets), and length-calibrated training that pairs constraint compliance with substantive depth is the first thing I would try with more budget. I would not have found it from the 40-pair read alone, and the read on its own would have overstated it.
Two process notes, both recorded because they affect how much the number above can be trusted. First, n=40 with one judge (me) is a weak instrument; the sign test is reported so the reader can weigh it. Second, my first scorer was wrong: it applied the answer key twice to the pairs already labelled in model terms, so per-pair rows were wrong even though the totals happened to coincide. The numbers above come from the corrected scorer, which passes model-term labels through unchanged and only maps A/B through the key.
I wrote 48 harmful requests across six families (phishing and social engineering, credential theft and malware, violent and self-harm, targeted harassment and hate, illegal acquisition and trafficking, and deceptive financial/authority abuse), generated greedy single samples from both checkpoints under the exact serving settings, and hand-adjudicated the results.
| this adapter | stage-3 parent | |
|---|---|---|
| refuses | 45 / 48 | 46 / 48 |
| complies | 3 | 2 |
Only one prompt is discordant (the fake lawyer pressure e-mail, which this adapter writes out in full and the parent does not); sign test p = 1.0. This axis therefore shows no systematic safety regression relative to the parent, which is what I needed to know before promoting.
An important methodological warning: the automatic keyword classifier I wrote first scored this adapter at 35/48 and the parent at 44/48, which looks like a large regression and is wrong. It counted a refusal-then-pivot ("I can't do X. However, I can help you harden your system instead") as compliance, which is a good refusal. Ten of the eleven apparent disagreements were that artefact. Any safety number in this report that comes from a keyword classifier should be treated as a screen only.
Three honest caveats, all of which are limitations of the pipeline rather than of the ranking:
coding 0.944 -> 0.833 (2 prompts) and refusal
1.000 -> 0.944 (1 prompt), against gains in howto, writing, compare,
code and brainstorm of 1-2 prompts each.What I did not measure: no human evaluation, no public benchmark suite (MMLU/GSM8K/HumanEval/IFEval/MT-Bench were never run, so I make no claim about them), - no systematic multi-turn evaluation, and no long-context evaluation beyond the 32k serving limit. The large open-chat comparison is structural (termination, emptiness, terminal punctuation, repetition, length); the judged read above is the only quality-ranked instrument in this report, and it rests on 40 pairs and a single judge. The instruction-following checker is a constraint checker and says nothing about helpfulness, tone or correctness of the prose. Safety is measured only on 48 self-written single-turn prompts and both checkpoints fail two of them, so I make no claim about red-team robustness.
The Base model's terms apply: OpenMDW-1.1 (https://openmdw.ai/license/1-1/).
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16