Assay reproducibility bundle — the seeded M / M' pair and a calibrated band

This is not a model you should generate text with. It is the fixture that makes Assay's model-substitution experiments reproducible offline, in seconds, with no training and no downloads of anything large.

• Code: github.com/NagaYu/assay
• Demo: NagaYu/assay-demo
• Receipt corpus: NagaYu/assay-receipts

Contents

Why M' has a permuted head

This detail matters more than it looks. A randomly-initialized GPT-2 is dominated by the residual copy of the last prompt token, so two independently seeded models both greedily echo that token — they are behaviourally identical, and a "substitution" between them is one nobody could notice in the output. Permuting the tied embedding does not help either: the echo circuit reads and writes through the same matrix, so the permutation cancels.

So M' here has its output head untied and permuted. It stays a well-formed model, it produces genuinely different text, and the substitution becomes something worth detecting. Anything that reads the emitted tokens — including Assay's generation-consistency check and the forgery it defends against — is meaningless without this.

Using it

from transformers import GPT2LMHeadModel
from huggingface_hub import snapshot_download

path = snapshot_download("NagaYu/assay-tiny-pair")
M  = GPT2LMHeadModel.from_pretrained(f"{path}/tiny_pair/M").eval()
Mp = GPT2LMHeadModel.from_pretrained(f"{path}/tiny_pair/Mp").eval()

The identical pair is also constructed on the fly by the library, with no download at all:

from assay.hooks import tiny_random_pair
M, Mp = tiny_random_pair(seed=0)   # bit-identical to the weights published here

Both are published because the copy here pins the fixture: if the constructor ever changes, these weights still reproduce the numbers in the paper-style README and the receipt corpus.

calibration.json

The tolerance band Assay's verifier compares against, measured on honest gpt2 traffic with injected numerical jitter, plus the sketch protocol parameters:

{
  "sketch": {"seed": ..., "layers": [3, 6, 8, 11], "n_positions": 6, "sketch_dim": 32},
  "calibration": {"mu": {...}, "sigma": {...}, "gen_mismatch_rate": ...},
  "verifier": {"deep_check_rate": 0.25, "alpha": 0.01}
}

gen_mismatch_rate is the rate at which an honest greedy provider's emitted token differs from M's own argmax at an inspected position — the null rate of the check that catches a forged receipt. It is calibrated rather than assumed to be zero, because numerical jitter can flip a near-tie.

The HMAC key in that file is published deliberately: without it the signatures in the receipt corpus cannot be checked. It is a demo key for a research prototype, not a secret, and not a TEE attestation key.

Intended use and limits

Intended for reproducing and probing substitution-detection experiments. These weights are random — M and M' have never been trained and their text is meaningless. Do not use them for anything other than the verification experiments they exist for.

Citation

@software{assay2026,
  title  = {Assay: per-request internal receipts for model-substitution detection},
  author = {Nagao, Yuta},
  year   = {2026},
  url    = {https://github.com/NagaYu/assay},
  note   = {Research prototype}
}