Task-specialized fine-tune of Laya (421M, ModernBERT-large backbone) for verifying v1.1 actor-oriented SPO assertions extracted from United States statutes and regulations. Given a source section and a candidate assertion (actor → relationship → target), it returns three calibrated noul (true/false probability) verdicts. It is designed to pair with XLL SPO Nemotron 4B, which extracts the candidate relationships. The verifier checks:
This is the v1.1 three-question actor contract — a breaking change from
the single claim_stated verifier previously shipped in this repository.
Campaigns citing thurstonpowers/xll-laya-verifier (checkpoint laya-2ep)
refer to that earlier revision, retrievable via git history.
The frozen question texts ship in questions.json (canonical sha256
d24107da8afca6260de1366b7a17e6a857b1e3a9e06b436ce9fffacfed993962) and are
byte-identical to training. The canonical JSON SHA-256 above hashes
json.dumps(questions, sort_keys=True, separators=(",", ":")).encode("utf-8");
it is not the raw-file checksum. The raw-file SHA-256 is
1de33a92f9a0a9b33a757d308620484cba6982980395d9b86c61d8ed13737469.
Both checks are shown below. State shape:
{
"source_identifier": "ct/regulations/sec-4-157-6",
"source_text": "...full section text...",
"candidate": { "subject": {...}, "predicate_text": "...", "object_literal": "...", "quote": {...}, "negated": false, "qualifiers": [], "participants": [] },
"reading": {
"subject": "The commissioner",
"predicate": "may order",
"target": "a pre-hearing conference",
"qualifiers": [],
"participants": []
}
}
reading uses resolved_to → raw → citation in that order, with
object_literal as the target fallback.
import hashlib
import json
from pathlib import Path
from huggingface_hub import HfApi, snapshot_download
from laya import load
repo_id = "thurstonpowers/xll-spo-verifier"
revision = HfApi().model_info(repo_id).sha # resolve once; retain this commit for reproducibility
snapshot = snapshot_download(repo_id, revision=revision)
question_bytes = (Path(snapshot) / "questions.json").read_bytes()
assert hashlib.sha256(question_bytes).hexdigest() == "1de33a92f9a0a9b33a757d308620484cba6982980395d9b86c61d8ed13737469"
questions = json.loads(question_bytes)
canonical = json.dumps(questions, sort_keys=True, separators=(",", ":")).encode("utf-8")
assert hashlib.sha256(canonical).hexdigest() == "d24107da8afca6260de1366b7a17e6a857b1e3a9e06b436ce9fffacfed993962"
agent = load(snapshot) # model and questions from the same pinned revision
result = agent.predict(state, questions) # all three questions, one forward pass
print({k: round(v["noul"], 3) for k, v in result["answers"].items()})
Runs on CPU. Verification bands used in the XLL campaign: fail if any noul < 0.5, flag if any < 0.65, else pass (heuristics, not calibrated decision thresholds).
Fine-tuned on the DGX Spark fleet with thermally paced full-parameter training (pause 87°C / resume 80°C), micro-batch 1, effective batch 64:
| Setting | Value |
|---|---|
| Rows | 61,279 train (34,193 unique) / 2,104 dev |
| Epochs | 1 completed of 2 planned (operator early stop after epoch 1; checkpoint then calibrate + evaluate only) |
| Calibration | 4,767 items, source-group disjoint |
| Finished | 2026-10-12 |
Training data: thurstonpowers/xll-spo-verifier, v1.1/laya/ (train/dev
splits disjoint by CFR part group; exact payload dedup; contradictory gold
rejected).
Calibrated held-out evaluation (DEV_EVAL.json, 6,312 items,
source-group disjoint):
Calibration caveat: the fitted noul temperature clamped at the top of its [0.5, 5] range, and the laya runtime reports some checkpoint temperature entries outside the valid interval (substituted with 0.5). Treat confidence values as approximately calibrated; the recommended decision rule is threshold banding, not raw-confidence interpretation.
See WEIGHTS_PROVENANCE.json for weight hashes, question-set binding, and
the verification chain.
Training and evaluation data: thurstonpowers/xll-spo-verifier. Use the model-specific contract and splits described in the dataset card.
For the current actor-relation checkpoint, use the v1.1/ files, not the
legacy provision-oriented root files. The dataset includes exact model-bound
splits, hashes, original reports and replication code.
convaiinnovations/laya