Transcript cleanup tiny

An experimental English KEEP/DELETE tagger for an optional cleanup step after transcription. The model proposes word deletions. The shared recipe runtime checks those proposals and renders edits from the original text.

The frozen pilot matches 39 of 48 synthetic held-out references. It preserves all 24 cases that should remain unchanged and correctly cleans 15 of the 24 cases that need edits. The nine misses remain unchanged. These results do not establish reliability on real dictation.

This repository contains the weights, tokenizer, and training manifest. Runtime code and experiment evidence live in the recipe's cleanup/ folder. No ASR application or streaming integration is included.

Model

Weight SHA-256: 1d7ec6c86525c1d1072a1889b23493df20fc9f59d6c9e0f412e3b5a3fa086208.

Use the guarded runtime

Install the Python dependencies:

python -m pip install torch==2.15.0 transformers==5.22.2 huggingface-hub==2.4.0 safetensors==0.8.0

Download the recipe code and model, then keep the Tagger instance resident between utterances:

from pathlib import Path
import sys

from huggingface_hub import snapshot_download

recipe_path = snapshot_download(
    repo_id="arach/parakeet-coreml-recipe",
    revision="98fcbb8ff7595c3b4a7abeb3d3ef8671f28c7426",
    allow_patterns=["cleanup/cleanup.py", "cleanup/edit_policy.py"],
)
sys.path.insert(0, str(Path(recipe_path) / "cleanup"))
from cleanup import Tagger

model_path = snapshot_download(
    repo_id="arach/transcript-cleanup-tiny",
    # Set revision to a published model commit for reproducible use.
)
tagger = Tagger(Path(model_path))
result = tagger.run("Um, the wicker basket is beside the radiator.")
print(result["output"])
# The wicker basket is beside the radiator.

The reported results require the complete Tagger pipeline: source alignment, the frozen threshold, eligibility guards, and the original-text renderer. A generic Transformers token-classification pipeline omits these steps. In the held-out evaluation, the model proposed deleting one quoted uh; the guard blocked that proposal.

The intended experiment is an optional pass on a completed utterance, with the original transcript retained. Speculative streaming is not implemented. See the recipe instructions for the CLI and result fields.

Edit policy and limits

Eligible edits remove unquoted standalone um or uh, or one copy of an adjacent repeated word. Repeated words must be separated by horizontal whitespace; comma-separated repetitions are ineligible in this version. The renderer retains at least one copy and repairs neighboring separators. It can capitalize the first surviving letter.

The policy leaves grammar, hedges, emphasis, explicit self-corrections, and incomplete speech intact. Guards protect recognized quoted text, code, URLs, digits, negations, underscore identifiers, and camelCase. They do not recognize every name or literal. The model can still make incorrect eligible deletions on unseen inputs.

Inputs over the context limit or with unsafe source alignment return the original text with a fallback reason. They are not truncated. Edit offsets are Python character indices. The edit list records lexical deletions; separator and capitalization repairs are implicit, so the list is not a complete patch format for an application.

A known prefix-capitalization discrepancy remains: deleting the first go in go go home. can render Go home., while training-label derivation permits that capitalization change only after a prefix filler. The frozen holdout does not exercise this case. The policy and label derivation need alignment before a subsequent experiment.

Training and evaluation

All examples are assistant-authored synthetic text. There are no human-approved labels or real ASR captures. The development set contains 160 pairs from 80 families, including 80 unchanged pairs. A family-level split assigns 128 pairs to training and 32 to validation. The model trains for 12 fixed epochs with seed 23, batch size 8, learning rate 0.0003, and DELETE loss weight 4.

Validation selects the threshold by minimizing cases that lose required words, then changed unchanged cases, then reference mismatches. Ties prefer the higher threshold. The selected threshold is 0.99. The weights, threshold, and renderer were frozen before evaluation on the separate 48-case pack.

The tagger's nine misses comprise two unpunctuated fillers, five eligible repetitions below the threshold, and two comma-separated repetitions outside renderer scope. Rules clean more cases but incorrectly shorten grammatical had had. The Qwen adapter was trained earlier under a broader cleanup policy and runs in MLX; this is a comparison of practical prototypes, not an isolated architecture comparison.

Reference matching normalizes outer whitespace and quote typography. Unchanged-case preservation uses exact string equality. Required-word loss is a lexical subsequence diagnostic, not proof of semantic fidelity. Four frozen preserve annotations contain cleaned phrases absent as contiguous input spans. The comparison report accounts for this diagnostic artifact; raw protected-span failure counts must not be interpreted as lost source text.

An audit found no exact or normalized duplicate sentences and no shared six-word sequences between development and holdout data. Four similar templates remain. Excluding those templates gives 35/44 matches for the tagger, with all 21 unchanged cases preserved. Synthetic independence remains limited.

The experiment folder contains the training source, data, and evaluation harness. Recorded results include the comparison, raw evaluations, and length probes. Training details and hashes are also recorded in tagger_manifest.json.

Timing scope

Measurements use an Apple M1 with 16 GiB RAM, PyTorch CPU inference, four threads, and batch size one. Five serial repeats per held-out case produce 240 timing observations, not 240 independent quality cases. Five warmups are excluded. Times include tokenization, inference, guards, and rendering; they exclude model loading, ASR, and delivery.

Fresh-process initialization, including Python library imports, took 13.50 seconds in the holdout run and 6.07 seconds in the length run. Warm latency assumes a resident model. Overlapping length probes with 20, 71, and 191 body words took 4.81–86.57 ms, 4.41–63.21 ms, and 9.31–48.11 ms respectively. Workstation scheduling varied substantially; these observations are not a latency guarantee or controlled scaling curve.

Native startup, Core ML conversion, process memory, energy, thermal stability, and concurrent ASR latency or memory have not been measured. Real Parakeet transcripts with human-approved minimal edits and a fresh recording-session holdout are needed before application integration.

Licenses

These weights derive from the Apache-2.0 BERT-Tiny base and are released under Apache-2.0. The shared recipe code is MIT licensed. Parakeet artifacts have a separate CC-BY-4.0 license; that license does not apply to these cleanup weights.