gpt-oss-120b Transcript Formatter — LoRA adapter (SFT L0–L5 + RL-aligned)

A LoRA adapter for openai/gpt-oss-120b that turns a raw speech-to-text transcript into a clean, formatted transcript — fixing punctuation, casing, fillers, disfluencies, homophones, ITN, garbled proper nouns / phonetic URLs-emails, and spoken layout commands — while preserving the speaker's words and passing through anything that is already correct.

This repo ships only the LoRA adapter (~hundreds of MB), not the merged 120B. Merge it on demand against the public base — see Usage. The teacher's distilled 20B student (≈ same accuracy, no reasoning, cheaper) is the recommended deployment target.

Model details

Training pipeline

A layered curriculum (each stage continued-SFT from the previous, with replay of all prior categories to prevent forgetting), then RL alignment:

21 categories total. Full dataset: Akash-Sakala/transcript-formatter-curriculum.

Results

Held-out evaluation over 2,384 rows across all 21 categories, scored with a two-round protocol: deterministic Round 1 (EM / ContentOK / CER), then an independent Claude judge re-adjudicates the Round-1 fails to credit cases where the model is correct but the gold label is unrealistic ("adjusted accuracy").

Per-layer adjusted accuracy: L0 100% · L1 99.9% · L2 99.8% · L3 100% · L4 100%. Only 2 genuine residual errors in 2,384 rows (one ITN edge case, one stacked disfluency). No forgetting across layers; no hallucination.

EM understates quality for the layout categories (many valid paragraph/list renderings); read ContentOK / adjusted accuracy / CER, not EM.

Usage

⚠ The system prompt is required (the model was trained with it verbatim) — see merge_and_infer.py for the exact text and a runnable example.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "openai/gpt-oss-120b", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "Akash-Sakala/gpt-oss-120b-transcript-formatter-lora")
tok = AutoTokenizer.from_pretrained("openai/gpt-oss-120b")

SYSTEM_PROMPT = open("merge_and_infer.py").read()  # copy the verbatim prompt from there
messages = [{"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": "i spoke to aisha she confirmed the PR will be merged by friday"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=4096, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
# → "I spoke to Aisha. She confirmed the PR will be merged by Friday."

Merge once for serving (vLLM/SGLang):

# Option A — PEFT
merged = PeftModel.from_pretrained(base, "Akash-Sakala/gpt-oss-120b-transcript-formatter-lora").merge_and_unload()
merged.save_pretrained("./gpt-oss-120b-transcript-formatter-merged")

# Option B — Tinker cookbook (handles sharded / quantised merges for vLLM)
from tinker_cookbook import weights
weights.build_hf_model(
    base_model="openai/gpt-oss-120b",
    adapter_path="./adapter",            # this repo, downloaded locally
    output_path="./merged",
    dtype="bfloat16",
)

Hardware: gpt-oss-120b is a 120B MoE — use a multi-GPU / large-VRAM host (or an MXFP4-quantised serving stack). For cheaper deployment, use the distilled 20B student (direct-final, no reasoning).

Intended use & limitations

• Use: post-processing ASR/Whisper output into readable text (dictation, meeting notes, voice memos, emails). Greedy decoding (temperature 0) for deterministic output.
• Preserves words. It edits format, not meaning — it does not paraphrase, summarise, answer, or invent content; ambiguous cases pass through unchanged.
• Limitations: English only; rare/novel proper-noun spellings can be missed; paragraph segmentation of free-dictated emails is subjective; not a general assistant (it only formats transcripts).
• License: Apache-2.0, inherited from the base openai/gpt-oss-120b.

Citation / provenance

Trained with the Tinker API. Teacher checkpoint: 85613875 (RL-final). LoRA rank 16. See the dataset card for the full curriculum.