Downstep — Japanese pitch accent for compounds your dictionary has never seen

Getting 東京大学 right is a lookup. The hard part is 新型感染症対策本部.

Downstep is a 35M-parameter encoder that predicts the accent nucleus and accent-phrase boundaries of a Japanese noun compound from its morphemes. It reports a calibrated confidence and, below a threshold, abstains — returning the existing front end's answer unchanged — so it can be added to an OpenJTalk-based TTS pipeline without replacing it.

Code, benchmark builder and evaluation: https://github.com/NagaYu/downstep

---

What this model is for, and what it is not for

Use it for the fallback path: compounds that are not entries in your TTS dictionary, where a rule-based front end has to compose the accent from parts.

Do not use it to replace dictionary lookup. On compounds that are in the dictionary this model is clearly worse than OpenJTalk (81.3% vs 96.4%), and worse than the dictionary itself (100% by definition). That is not a defect — the dictionary is right and there is nothing to add. The shipped adapter tries the dictionary first for exactly this reason.

This is a text-only model. It reads a surface string and a kana reading; it has never seen speech. Accent models trained on thousands of hours of audio are solving a different problem.

---

Results

Evaluated on 5,414 held-out compounds, scored identically in both conditions. Gold accents come from NAIST-jdic and UniDic 2.1.2. Metric: accent-nucleus exact match.

+40.7 points on dictionary-unlisted compounds. On the hard subset — compounds whose final morpheme never appeared in training — 73.3% vs 40.9%, so the gain is not memorisation. OpenJTalk also wrongly splits an unlisted compound into two accent phrases 23.3% of the time; Downstep does so 2.2% of the time.

The "not in dictionary" condition is produced by ablating the compound's own entry at query time so the gold stays real. It is a held-out-entry simulation, not an observation of genuinely novel compounds — see Limitations.

Calibration. Temperature scaling fitted on dev: ECE 0.048 → 0.011 (T = 1.498).

n-best. top-1 81.3%, top-3 98.0%, top-5 99.7%. On words whose dictionaries attest more than one accent, top-1 is 61.2% but top-3 is 98.0%.

Speed. 2.0 ms/word (PyTorch, CPU). ONNX int8: 1.3 ms at batch 1, ~3× faster than fp32.

---

Files

vocab.json is not optional. Feature id spaces are learned from the training corpus, so a mismatched vocab does not raise — it silently predicts nonsense.

The int8 build's decisions are identical to fp32 (nucleus argmax and boundary sign agree 100%), but its logits deviate enough to fail a 5e-02 tolerance. Since the abstention gate thresholds a confidence, re-calibrate before using int8 with a threshold rather than inheriting the fp32 one.

There is no GGUF. llama.cpp dispatches on general.architecture and has no downstep entry, and its token-ids-in / vocabulary-logits-out interface does not describe this model, which takes 13 parallel feature streams and returns three per-mora logit vectors. ONNX is the export that actually runs.

---

Usage

pip install "downstep[all] @ git+https://github.com/NagaYu/downstep"

from huggingface_hub import snapshot_download
from downstep.adapters.openjtalk import DropInAdapter

path = snapshot_download("NagaYu/downstep")
adapter = DropInAdapter.load(path)          # dictionary first, model for the rest

# VOICEVOX-shaped accent phrases
adapter.accent_phrases("新型感染症対策本部を設置した")

# NJD features — same keys as pyopenjtalk.run_frontend(), swap it in directly
adapter.njd_features("新型感染症対策本部を設置した")

# HTS full-context labels
adapter.full_context_labels("新型感染症対策本部を設置した")

For VOICEVOX, substitute adapter.accent_phrases(text) where the engine builds its AudioQuery accent phrases. For Style-Bert-VITS2, point its pyopenjtalk.run_frontend call at adapter.njd_features.

A trap worth knowing. In HTS full-context labels the accent field F2 writes 平板 (flat, no fall) as F2 == F1, not as 0 — so F2 == F1 is ambiguous between 平板 and 尾高 (nucleus on the final mora). Writing a literal 0 there silently corrupts every flat-accent word, and reading F2 naively scores OpenJTalk at 70% where the truth is 94%. The adapter converts in both directions and raises AmbiguousF2Error rather than guessing.

---

Training

• Data: 71,544 compounds from NAIST-jdic, each decomposing into ≥2 morphemes whose readings compose to the compound's reading under 連濁 (rendaku) and 促音化. Decomposition uses fugashi + UniDic — deliberately a different dictionary from the gold, so the parts handed to the model are not derived from the answer.
• Architecture: 10-layer pre-LN transformer encoder, d_model 512, 8 heads, d_ff 2048. 13 per-mora integer feature streams (mora identity, position, morpheme boundary, word origin, each component's own dictionary accent and sandhi class, …) summed as embeddings. Three per-mora heads: boundary, nucleus, heiban.
• Objective: boundary BCE + a per-phrase softmax over (each mora position, plus a no-nucleus option). That single softmax per phrase is the training-time counterpart of the decoder's hard constraint that an accent phrase carries at most one nucleus.
• Run: 12 epochs, batch 256, AdamW 3e-4 with cosine decay, 82 minutes on Apple MPS.
• 25% of training items are constructed two-phrase examples (two compounds joined by a case particle), because every real benchmark compound is a single accent phrase and the boundary head would otherwise see only negative examples.

---

Limitations

• No human annotation exists anywhere in this project. Every label is a dictionary entry. The compounds this model is built for are precisely the ones no dictionary contains, and their accent can only be settled by asking native speakers. The repository ships the full annotation protocol, multi-annotator format, agreement statistics and an extractor that emits novel compounds unlabeled — but nothing here has been checked by a person.
• The unlisted condition is simulated. Held-out-entry ablation on lexicalised compounds is what makes real gold available; it is not an observation of genuinely novel compounds.
• Per-word non-degradation is expensive. At full coverage the model makes 256 of 5,414 words worse while making 2,462 better (~10:1). Driving individual regressions to zero requires abstaining on ~93% of items, which scores 42.2% instead of 81.3%. In aggregate the combined system is at or above the baseline at every coverage level; individual words are not. Choose the threshold deliberately.
• Proper nouns are 66% of the benchmark. Place and person names are a different, more arbitrary problem. They are always reported as their own stratum; the common-noun result rests on 2,033 test items.
• Boundary F1 is not a hard task here (dev reaches 1.000) because the only positive boundary examples available offline are constructed. The informative boundary number is the wrong-split rate above.
• hard is 818 items and its strictest variant is 98. The generalisation result is real but rests on a small sample.

On variation

Standard (Tokyo) accent is the default because that is what the two gold lexicons encode — a statement about available data, not about how anyone should speak. Regional and generational accent patterns are ordinary variation, not error. Where the dictionaries attest several accents for a word, all are counted correct, and n-best exists for the same reason. Non-Tokyo systems belong in separate adapters; that interface ships with no invented accent data behind it, because no licensed non-Tokyo accent resource was available.

License and attribution

Code and weights: Apache-2.0.

Trained on data derived from UniDic 2.1.2, © The UniDic Consortium (NINJAL), used under the BSD 3-clause option of its GPL/LGPL/BSD tri-licence, and from the NAIST Japanese Dictionary (naist-jdic), © Nara Institute of Science and Technology, BSD 3-clause. Neither dictionary is redistributed here.