PlayableNotes

Benchmark · optical music recognition (OMR) · piano

Sheet music scanner accuracy benchmark: 7 apps, 17 piano uploads

We ran the same 17 uploads of printed piano music through seven sheet music scanner apps and scored each one’s MusicXML export bar by bar against checked answer keys. PlayableNotes had the highest average on these uploads, 97.7% of bars exactly right, then Soundslice 89.4%, Songscription 78.7%, Halbestunde 73.0%, Scan2Notes 71.7%, PlayScore 2 37.5% and Sheet Music Scanner 17.7%. PlayableNotes’ first, blind reading of the set scored 82.3%, behind Soundslice; the 97.7% came after two days of development guided by what it got wrong, each change tested on other pages first (all four readings are below). The uploads, answer keys, every app’s raw export and the scorer are public on GitHub.

Checked MethodRun and published by PlayableNotes, one of the seven apps tested. Read it as a vendor’s benchmark, and check it: every file is public.

Figure 1

Bars read exactly right, all 17 uploads

  1. PlayableNotes 97.7%
  2. Soundslice 89.4%
  3. Songscription 78.7%
  4. Halbestunde 73.0%
  5. Scan2Notes 71.7%
  6. PlayScore 2 37.5%
  7. Sheet Music Scanner 17.7%
Share of the answer key’s bars matched exactly, in order (every pitch, start and length in the bar; extra bars in an export are not counted against it), averaged over the 17 uploads; higher is better. Run 16 to 18 November 2026 by PlayableNotes, a vendor; the phone photos are synthetic. PlayableNotes’ lead over Soundslice is 8.3 points: 95% interval 2.0 to 16.3 points resampling uploads, 1.5 to 17.4 resampling the 10 pieces (ci.py); better on 10 uploads, tied on 5, worse on 2; 5.1 points without the voice-and-piano upload (H). These describe this set of pages, not printed music in general. Every number on this page comes from the repository at commit 571f9a8.

Four full-set PlayableNotes runs

The other six apps were each run once. PlayableNotes was run four times as it was developed, and all four readings are in the repository. The first, through the website, was the first look at these uploads; what it got wrong guided later changes, which were tested on other pages before this set was read again. So the later readings are less blind than the competitors’ single runs: the first, blind reading scored 82.3%, 7.1 points behind Soundslice (95% interval -19.4 to +3.0).

Each run’s exports are in its folder under results/; the tables below use the 18 November release. Partial reruns of a few uploads, made while diagnosing errors, aren’t shown.

At a glance

Highest average here
PlayableNotes, 97.7% of bars exactly right; Soundslice 89.4% (lead 8.3 points, 95% interval 2.0 to 16.3); Songscription 78.7%. PlayableNotes’ first, blind reading scored 82.3%.
Phone photos
Good photos: PlayableNotes 98.4%, Songscription 96.7%, Soundslice 92.8%. Bad photos: PlayableNotes 93.8%, Soundslice 87.1%.
Test set
17 uploads made from 10 piano pieces: 9 public-domain scans and an engraved lead sheet, 6 synthetic phone photos, 1 low-resolution screenshot; 469 bars scored.
Scoring
One scorer for every tool: each MusicXML export against a checked answer key, bar by bar.
Public test set
OLiMPiC (Charles University): TEDn 14.05% on 300 systems, against 18.40% for the authors’ model on all 1,493.
Check it
Uploads, answer keys, every raw export and the scorer are on GitHub.

Table 1

Accuracy by image type: scans, phone photos, screenshot

AppAll 17Scans (10)Good photos (3)Bad photos (3)Low-res screenshot (1)Bars exact, pooledNote F1
PlayableNotes97.7%98.5%98.4%93.8%100%458 of 4690.997
Soundslice89.4%89.1%92.8%87.1%89.5%419 of 4570.950
Songscription78.7%82.3%96.7%59.0%47.4%366 of 4690.895
Halbestunde73.0%72.8%73.4%74.8%68.4%332 of 4610.893
Scan2Notes71.7%70.2%73.4%76.2%68.4%324 of 4570.886
PlayScore 237.5%53.5%31.6%2.4%0%237 of 4690.777
Sheet Music Scanner17.7%29.4%2.4%0%0%110 of 4610.629

Bars read exactly right, mean per upload in each group; higher is better. Pooled: all bars counted together. PlayableNotes’ scored exports have 469 bars for the keys’ 469, so the rule that extra bars aren’t counted against an export doesn’t help it. The totals differ because the hymn page (G) also prints the end of the previous hymn: each app is scored on the 12-bar key (the hymn alone) or the 16-bar key (with the previous hymn’s end), whichever gives it the higher bars-exact plus note F1. Scoring everyone on the 16 bars puts Soundslice at 85.5%; on the 12 bars, PlayableNotes at 97.6% and Soundslice at 89.4%. The order is the same either way. Note F1: notes matched in order on pitch and length, 0 to 1. Scan2Notes and Halbestunde use the same recognition engine.

Songscription came close on the good phone photos (96.7%) but fell to 59.0% on the bad ones and 47.4% on the screenshot. PlayScore 2 read about half the bars on the scans and a third on the good photos, and almost none on the bad photos or the screenshot.

Table 2

Fingering, lyrics, chord symbols, repeats and navigation marks

AppFingering F1 (8 uploads)Lyrics F1 (5)Chord symbols F1 (lead sheet I)Repeats and endings (8)Navigation marks (lead sheet I)
PlayableNotes0.980.991.001.001.00
Soundslice0.950.910.941.000.00
Songscription0.590.981.000.880.50
Halbestunde0.000.900.940.550.50
Scan2Notes0.000.000.000.000.00
PlayScore 20.140.000.000.470.25
Sheet Music Scanner0.000.000.000.560.00

F1 from 0 to 1: fingering as (note, finger) pairs matched in order; lyrics as syllables matched in order, verse by verse; chord symbols matched in order. Repeats: the number of repeat signs and ending brackets in the export against the key’s, capped at 1 (counted, not matched in place). Navigation marks: the segno, To Coda, coda and D.S. al Coda on the Shenandoah lead sheet (I), the only upload with them, checked in the export, not in each app’s playback. Higher is better.

Soundslice’s exports had as many repeat signs and endings as the keys but none of the lead sheet’s four navigation marks, so software that plays the export would not jump back to the segno.

Table 3

Every upload, every app

UploadBarsPlayableNotesSoundsliceSongscriptionHalbestundeScan2NotesPlayScore 2Sheet Music Scanner
A Kuhlau, Sonatina Op. 20 No. 1 (scan, fingering)41100985466668822
B Schumann, The Merry Peasant (scan)219595100100954343
C Joplin, The Entertainer (scan)1610010010010094250
D Mozart, Sonata K. 545 (scan, ornaments)1910089475353475
E Chopin, Preludes 20 and 21 (scan)21100864343388633
F Beethoven, Für Elise (3-page PDF)10698969770718145
G Hymn, four verses (scan)16*10010094676700
H Foster, Beautiful Dreamer, voice and piano (scan)1210042100333300
I Shenandoah lead sheet, chords, D.S. al Coda (PDF)17100889410010094100
J Beethoven, Minuet in G, da capo (scan)3591979497867146
C2 The Entertainer, good phone photo16100100100100100310
C3 The Entertainer, bad phone photo168810094949400
G2 Hymn, good phone photo16*10083100676700
G3 Hymn, bad phone photo16*948369626700
A2 Kuhlau, good phone photo419595905454637
A3 Kuhlau, bad phone photo411007815686870
D2 Mozart, 800-pixel screenshot191008947686800

Bars read exactly right on each upload, %. Scans are A to J (I is a vector PDF engraved for the test); 2 is a good phone photo and 3 a bad one; D2 is an 800-pixel screenshot. The 17 uploads come from 10 pieces, 304 distinct bars. *The hymn rows are scored out of 16 bars or 12 (the hymn alone), whichever gives that app the higher score, so 83% and 67% there are 10 and 8 of 12. Sources, editions and licences: index.json.

PlayableNotes was not best everywhere: on the Minuet in G (J) Soundslice, Halbestunde and Songscription read more bars right; on the bad photo of The Entertainer (C3) Soundslice, Songscription, Halbestunde and Scan2Notes did; and on The Merry Peasant (B) Songscription and Halbestunde did. Check one: on C3, Soundslice got all 16 bars and PlayableNotes 14, reading a C as a B in bar 3 and two Ds as Cs in bar 12. See the photo, the answer key, Soundslice’s export and PlayableNotes’ export, or run python3 diffbars.py PlayableNotes C3.

How this compares with other published tests

  • omr-research.net, 2019. It found PlayScore 2 got 91.8% of notes right on piano scans, ahead of SmartScore X2 and PhotoScore. Here PlayScore 2 had a note F1 of 0.89 on the scans, close to that. The low figure on this page is bars read exactly right, a stricter measure (one wrong note fails the bar), and it includes phone photos, where PlayScore 2 read much less.
  • Scoring Notes, December 2024. Its review of six scanners found the machine-learning ones (Soundslice, Newzik) most accurate out of the box, and the mobile apps (PlayScore 2, Sheet Music Scanner) quickest but less accurate. Among the six other apps here it matches: Soundslice first, PlayScore 2 and Sheet Music Scanner last. PlayableNotes wasn’t in that review.
  • OLiMPiC, 2024. A public test set made by other researchers, scored with its authors’ own code: below.

Table 4 · a public test set by other researchers

OLiMPiC: piano systems from real scans, scored with TEDn

OLiMPiC (Mayer, Straka, Hajič jr. and Pecina, Charles University, ICDAR 2024) is 1,493 piano systems cut from real IMSLP scans of 19th-century songs, each with a hand-made MusicXML answer. It is scored with the authors’ own code: TEDn, a normalized tree edit distance between the predicted and the true MusicXML, roughly the share of the score that needs editing.

SystemTEDn fullTEDn (LMX)Symbol error rateTest systems
PlayableNotes (18 Nov 2026 release candidate)14.05%9.87%11.82%300-system sample
Zeus (dataset authors)18.40%14.85%17.72%All 1,493

Lower is better. The sample is 3 systems from each of the 100 test songs, fixed by a rule before any run; Zeus’s figures are from the paper (Table 3) on all 1,493, so the comparison is indicative. PlayableNotes ran as the release candidate of 18 November (build e03bf963, in the run’s run.json), the code released that day. The sample wasn’t held out: one change in that release (ignoring slivers of neighbouring systems at a crop’s edge) came from errors seen on it. The songs have been public for years, so a large model may have seen them.

All 300 predictions, with each system’s scores, cost and time, are in the repository’s olimpic folder, with the script that fetches the dataset and the authors’ scorer at a pinned commit.

Method

  1. The uploads. Nine US public-domain scans published 1864 to 1919 (Kuhlau, Schumann, Joplin, Mozart, Chopin, Beethoven’s Für Elise as a three-page PDF, a four-verse hymn, Foster’s Beautiful Dreamer for voice and piano, and Beethoven’s Minuet in G) plus a Shenandoah lead sheet engraved for the test, chosen to test several things at once: fingering, lyrics, chord symbols, repeats with endings, D.S. al Coda, ornaments, clef changes and an old page with show-through. Three were also made into a good and a bad phone photo, and one into an 800-pixel screenshot: 17 uploads from 10 pieces. A photo blurred until it can’t be read was run as a control and isn’t scored; the scored PlayableNotes run read 331 notes from it without a warning.
  2. The answer keys. One per piece, drafted by an AI agent working from the high-resolution source (a public reference edition was allowed as a starting point; the printed page wins wherever they differ) without seeing any app’s output, then rendered and checked measure by measure in several verification rounds (keys/BRIEF.md). The agents were Claude Opus 5.5, a different model family from the one PlayableNotes’ reader uses. A key audit checked every bar where three other apps agreed with each other against the key: none on the scans, and 7 on three photos and the screenshot, where Halbestunde, Scan2Notes (one engine) and Sheet Music Scanner read a treble-clef staff as bass clef; Soundslice’s reading of the clean scans confirms the key. They are in PlayableNotes’ text format, converted to MusicXML with its converter; the scorer reads the MusicXML.
  3. Running the apps. The six other apps read each upload once, as supplied, in their own apps on 16 November 2026, and the MusicXML export was saved before any editing. Soundslice asked questions on three uploads (D, D2 and G3) that had to be answered by hand, and its other offered options were left at their defaults; its first upload of C was interrupted before it finished and was run again. Sheet Music Scanner’s settings weren’t recorded (tools.csv). PlayableNotes’ readings are listed above; the scored one is its production pipeline of 18 November 2026, on our machine, exported with the same converter as the website’s download.
  4. Scoring. One scorer for every app, ours included: each export becomes a neutral list of notes, aligned to the key by bar. A key bar counts only when it is matched exactly, in order, by a bar of the export (every note’s pitch, start and length); extra bars in an export aren’t counted against it. Fingering is F1 over (note, finger) pairs in order; lyrics over syllables in order, verse by verse; chord symbols over the symbols in order.

Limits

  • A vendor’s benchmark. PlayableNotes chose the pages, made the keys (with an AI agent, in its own format) and ran the test. Its first, blind reading (82.3%) was behind Soundslice. The files are public so that others can check or repeat it.
  • A small set. 17 uploads from 10 pieces, chosen to test many things at once, not a random sample. Piano and voice-and-piano music only: no orchestral scores, guitar tab or handwriting. The lead over Soundslice is clear (95% interval 1.5 to 17.4 points resampling pieces) but not precise.
  • Single scored readings. Each score is one reading, and readings vary from run to run, for every app.
  • Familiar pieces. Für Elise, The Entertainer and K. 545 are among the best-known piano pages, and PlayableNotes reads with a large language model that may have seen them, as with OLiMPiC above; the hymn, the Kuhlau page and the engraved lead sheet are less likely to be familiar.
  • Synthetic photos. Made from the scans (perspective, curl, shadow, uneven light, blur, noise); real phone photos would be more convincing.

Check it yourself

The repository nmharmon8/sheet-music-scanner-benchmark has the 17 uploads exactly as each app got them (index.json describes each), the answer keys (MusicXML), every app’s unedited export, each tool’s plan and settings (tools.csv), the scorer (Python, standard library only) and every score per upload (scores.csv). python3 score.py reproduces these tables in seconds; python3 ci.py gives the intervals; python3 diffbars.py Soundslice H lists the bars an app got wrong on one upload. To test another app, run the uploads through it and add its exports. The numbers here are from commit 571f9a8. If you find a wrong bar, open a GitHub issue naming the upload and bar number (e.g. “C3, bar 12”), or write to support@playablenotes.com.

Frequently asked questions

Which sheet music scanner is the most accurate?

In this benchmark of seven scanners on 17 printed piano uploads, PlayableNotes had the highest average, 97.7% of bars exactly right, Soundslice 89.4%, Songscription 78.7%, Halbestunde 73.0%, Scan2Notes 71.7%, PlayScore 2 37.5% and Sheet Music Scanner 17.7%. PlayableNotes’ lead over Soundslice is 8.3 points (95% interval 2.0 to 16.3 points over the uploads). PlayableNotes read the set four times as it was developed: its first, blind reading scored 82.3%, behind Soundslice. PlayableNotes makes this page; the uploads, answer keys, every export and the scorer are public so the numbers can be checked.

Which app reads phone photos of sheet music best?

On the three good phone photos, PlayableNotes read 98.4% of bars exactly right, Songscription 96.7% and Soundslice 92.8%. On the three bad photos (angled, shadowed, another page in the frame), PlayableNotes read 93.8%, Soundslice 87.1% and Scan2Notes 76.2%; PlayScore 2 and Sheet Music Scanner read almost nothing. The photos are synthetic, made from the scans.

How was accuracy measured?

Each of the six other apps read each upload once in its own app, and one scored MusicXML export per app per upload was saved unedited; the exceptions are recorded (Soundslice needed hand answers on three uploads and one upload was re-run after an interruption; Sheet Music Scanner’s settings weren’t recorded). PlayableNotes read the set four times as it was developed; the scored entry is its 18 November release, and its first, blind reading scored 82.3%. One scorer compared every export with an answer key drafted by Claude Opus 5.5 agents from high-resolution sources, without seeing any app’s output, and checked measure by measure. A key bar counts only if a bar of the export matches it exactly: every note’s pitch, start and length.

Is this benchmark independent?

No. PlayableNotes ran it, and PlayableNotes is one of the seven apps. PlayableNotes also read the set four times as it was developed, and its first, blind reading (82.3%) was behind Soundslice (89.4%); the scored 97.7% came after changes guided by what that reading got wrong. That is why every file, including all four readings, is public: anyone can re-run the scorer on the same exports, or run the uploads through any app themselves. For a test set made by others, PlayableNotes was also scored on OLiMPiC with that dataset’s own scorer.

What are OLiMPiC and TEDn?

OLiMPiC is a public test set for reading piano music, from researchers at Charles University (ICDAR 2024): 1,493 piano systems cut from real scans, each with a hand-made MusicXML answer. TEDn is its scorer: a normalized tree edit distance, roughly the share of the score that would need editing. PlayableNotes scored 14.05% on a fixed sample of 300 systems; the authors’ own model, Zeus, scored 18.40% on all 1,493.

Why does PlayScore 2 score so low here when other tests rate it well?

Different measures. omr-research.net (2019) found PlayScore 2 got 91.8% of notes right on piano scans; here its note F1 on the scans was 0.89, close to that. But a bar counts here only if every note in it is right, and at that level PlayScore 2 read 53.5% of bars on the scans, 31.6% on the good phone photos and almost none on the bad photos and the screenshot. Scoring Notes’ 2024 review likewise found the machine-learning scanners (Soundslice, Newzik) more accurate than the mobile apps, which trade accuracy for speed.

Which scanners weren’t tested?

ScanScore, SmartScore, PhotoScore, Audiveris (and MuseScore’s import, which uses it) and ScoreLens weren’t run in this round. The uploads, answer keys and scorer are public so that anyone can add them: export each upload as MusicXML and run the scorer.

Is PlayableNotes better than Soundslice on every page?

No. Soundslice read the Minuet in G (J) and the bad photo of The Entertainer (C3, 16 of 16 bars against 14) better, and other apps beat PlayableNotes on J, C3 and The Merry Peasant (B) too: it was better than Soundslice on 10 uploads, tied on 5 and worse on 2. Soundslice also reads guitar tab and multi-instrument scores, which PlayableNotes doesn’t.

Read your own page and check it.

New accounts get 50 free reading credits, about 7 to 10 piano pages. Compare the reading with your page, then listen, slow it down and practise hands apart.

Read a page free → Or open a free library piece

50 free reading credits. No card needed, no subscription.

The score PlayableNotes read from a page of Bach’s Prelude in C: title, composer and the first two lines, engraved and ready to play.