Legato 2

Full-page optical music recognition. A detector finds the systems on a page and a transcriber reads each one in turn, conditioned on the ABC it has already produced for the systems above it, so the notation stays consistent down the page.

Both halves live here: model.safetensors is the transcriber, segmenter.pt the YOLO line detector. They are versioned together because a change to either one changes the result.

Usage

from legato.pipeline import Legato2

omr = Legato2.from_pretrained("guangyangmusic/legato-2", device="cuda:0")
abc = omr("page.png")

Decoding follows the paper by default: greedy (beam size 1), repetition penalty 1.1, maximum length 2048; segmentation uses confidence 0.7 at image size 640. All of these can be overridden per call, for example omr("page.png", beam_size=10, yolo_conf=0.5).

A PDF can be transcribed from the command line, where it is first rendered to one tall image so a multi-page score is read as a single piece:

python scripts/run_legato2.py --image_path score.pdf --model_path guangyangmusic/legato-2

Checkpoint

legato-segment-bytes-tokenizer-lr-3e-4-cosine-e5-b32-v4096: byte-level tokenizer with a 4096-entry vocabulary, learning rate 3e-4 on a cosine schedule, five epochs at batch size 32.

Evaluation

OMR-NED, lower is better. Scored with our musicdiff fork (5.2) on music21 9.9.1 and converter21 4.0.1; the ABC is converted to MusicXML with abc2xml followed by a MuseScore 3.6.2 rewrite, which is part of the measurement and shifts the figures by one to two points if skipped.

License

The two files in this repository are licensed separately:

Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.