Can AI read and write sheet music (now)?

Besides working in IT, I have been singing in choirs for quite a while now. Anyone who has done the same will know what a well-used choir score looks like after a few months of rehearsals: pencil notes from several rehearsals and possibly several singers, breath marks, dynamics, a crossed-out note, an arrow that moves a syllable to a different note. Add a print that was not very crisp to begin with, and after a while it can get really hard to see what is important on the page.

Opinions on how far AI has come vary widely, and they depend a lot on who is talking: as developers we mostly talk about writing code, and in the media it is more about where we are on our path to AGI. Both discussions tend to stay rather abstract, so I like to check the progress on something small and concrete that I care about. In my case, I wanted to give a scanned choir score to a model and let it produce a clean, editable score in MuseScore.

To be clear about which AI I mean: specialized software for this has existed for some time. So-called optical music recognition (OMR) tools are built to turn printed sheet music into notation, and they are probably still the better choice for a clean print (but they usually come with a sizable license fee of their own). My question was a different one: can the general-purpose language models that basically everyone has access to today (the same ones we use for writing code) do this job, including the handwritten changes and the judgment calls that come with them?

About half a year ago I tried exactly that, and it just didn’t work. In September I tried again, and it did.

In this article I would like to share a snapshot of that difference. It is not a benchmark, as the tasks and the inputs were not identical. It is rather an example of a task for general language models that I had filed under “impossible (for now)” and that suddenly works.

Why transcribe a choir score at all?

The most annoying scribbles are usually not dynamics, but changes to the text underlay: which syllable of the lyrics is sung on which note. A conductor decides that the tenors should split a word differently than the print suggests, or that one voice sings a vowel over several notes where the others already move on. That ends up as a pencil arrow or a handwritten syllable under the staff, and in the next rehearsal half the section has forgotten about it.

One solution is to just transcribe the whole thing afresh and include all the adjustments directly in the new score. That fixes the readability, but there is a second benefit, which on its own is already a good reason to transcribe a score: once the music is in a notation program, it is really easy to export MIDI or MP3 tracks for each individual voice. Everyone can then practice their own part at home, at their own pace.

The catch: transcribing takes time. Entering four voices, lyrics and all markings note by note is tedious busywork, exactly the kind of work that I, as a developer, have gotten used to handing over to a model.

March 2026: can a language model read sheet music?

The setup when I tried this for the first time in March was fairly simple: VS Code with GitHub Copilot in agent mode, and an open-source MCP server for MuseScore (mcp-musescore by George Chen). MuseScore is a free notation program, and the MCP server connects to a small plugin inside it, so that the agent can move a cursor through the score and add notes, rests and lyrics. As models I used Claude Sonnet 4.6 and GPT-5.4.

The small tasks worked. Adding a scale to an empty score was no problem, and neither was undoing it. When I asked for “Alle meine Entchen” (a German nursery rhyme everyone can sing), the model confidently wrote the melody from memory; the last bar was wrong. A small thing, but a nice reminder that language models (at the time) remained just that, language models.

Then the real task: a page of a choir score for eight voices, which I wanted as a piano reduction for rehearsals. A clean print this time, no scribbles. Both models read the text without any problems, recognized the tempo marking and the number of voices, and one even identified the psalm the lyrics were taken from. Great!

The notes, however, were a different story:

• Sonnet read the time signature wrong until I corrected it, and decided that the piece was in D minor (it was not).
• It then admitted in its reasoning that it could not read the exact pitches, and instead made some “reasonable assumptions”: the Latin text says “qui descendunt” (“who go down”), so it wrote a line that goes down. The actual line in the score does not.
• GPT-5.4 spent about nine minutes and 35 tool calls on it, including six undos and a lost connection. It filled the score with chords roughly derived from what it could make out and, to its credit, said so and asked for a sharper image.

To sum it up: they read the words around the music, but not the music itself. The result was not usable, and I filed the whole idea under “not yet”.

September 2026: same idea, worse scan

Six months later I picked up the same project again, this time with Claude Opus 5.5 in Claude Code, and with the same MuseScore MCP server as in March. The input was, if anything, harder: a scan of “Höstens sjö”, a Swedish choir piece by Gauthier Fleutot, taken from my own choir folder, slightly skewed, with four voices squeezed onto two staves and several handwritten corrections to the text underlay.

I asked for the first seven bars. The model did not start by entering notes. First, it looked at what the MCP server could do and noticed that it could neither write two voices on one staff nor set a key signature, ties or dynamics. Instead of improvising around that, it asked me to set up a score with one staff per voice and told me which markings I would have to add by hand afterwards. It then entered one voice as a test, read it back from MuseScore to check pitches and durations, and only then wrote the remaining three voices, the lyrics and the tempo. Every note it had entered matched what it had read from the scan, and the entry itself took seconds.

It even went one step further: it pointed out that the missing features (ties, multiple voices, dynamics, key signatures) could be added to the plugin, and offered to do that itself. By now I have seen this several times when trying things that a package just can’t do yet: instead of stopping at the limitation, the model suggests a fix or an improvement, because it could write it itself. I didn’t follow that path (yet), though: this was more of an experiment to understand what the models are capable of than an actual need for a transcription (I had already done that by hand half a year ago). So instead of extending the tooling, I wanted to see how far I would get with what is already there.

What about the rest? Computer use

For everything the MCP server cannot do, I tried a second approach: letting the model use MuseScore like a person would, via screenshots, mouse and keyboard (computer use in Claude Cowork). In about 15 minutes of actual work (roughly 190 steps) it added the key signature, dynamics, breath marks and ties, entered the lyrics for all four voices including the handwritten changes, and transcribed the next five bars from the same scan. After another three minutes, it had created a piano reduction using MuseScore’s own tools.

The model knew MuseScore’s workflows surprisingly well (keyboard note input, palettes, lyrics mode, the undo history). Most of its trouble came from the control layer rather than from the music: text typed in the background came out garbled, and some key combinations just didn’t arrive while the model was working in the background (giving it full control of the screen fixed that). In both cases, the model noticed the problem itself and corrected it without me having to point it out.

The two approaches differ mostly in speed and reach. Right now, computer use still feels slow compared to a direct technical connection like the MCP server. But an MCP server only ever offers what someone has implemented for it, which is often just a fraction of what the full application can do (and that is not specific to MuseScore). Computer use does not have that limitation: whatever a person can do in the program, the model can do as well. With the tooling improving and computer use getting faster along with it, I would not be surprised if it ends up being the more practical option in the long run.

Just look closer?

What impressed me most was not the result itself, but how the model got there. In March, the models looked at the whole page, could not make out the details and then guessed. This time, the model treated the bad scan as a problem to solve.

It cut the scan into small pieces bar by bar and enlarged them. It wrote a small Python script that finds the staff lines by counting dark pixels per row. Because the scan was slightly skewed, it measured those lines separately along the page instead of assuming they were straight. For each note it then compared the position against the measured lines, and where that was still ambiguous, it drew a grid of helper lines over the image and looked again. As a final check it asked itself whether the resulting chords made musical sense, and after entering the notes it read them back from MuseScore and compared them with the scan. So when it could not see something, it found itself a way to see it.

To check that this was not just luck with an easier input, I gave the model the two screenshots from March again. The staff lines in those images are only about four to eight pixels apart, which explains why a quick glance was never going to work. With the same measuring approach it read the notes, explained how the harmonies move and noticed something neither of the old sessions had spotted properly: in one of the screenshots, half of the eight voices simply double the other half an octave lower. For a piano reduction that is the kind of insight that makes the job a lot easier.

What actually changed?

Half a year ago I had a fairly simple explanation for the failure: music is hard to represent as plain text, and in the end, plain text (or rather tokens) is what a language model works with internally. So why would a model that is built around language ever read a score properly?

To be honest, I don’t know what exactly has changed since then. It could be the tooling: the model now creates any additional tooling itself to better solve the task. It could be the training data: the model suddenly seems to understand harmony well enough to use it as a sanity check, which I would not have intuitively expected from a “language” model.

So it’s flawless now?

No, and I think it is important to say that. There were mistakes along the way:

• A long curved line in the scan can mean two different things in sheet music, and it picked the wrong one. Only after I explained how the text is distributed across the notes did it correct that.
• For one note it was unsure, and instead of hiding that, it told me.

In March, guesses were presented as results. This time the uncertainty was visible, and the mistakes were found and fixed, mostly by the model itself.

Final thoughts

To bring it back to where I started: half a year ago I put “AI transcribes my choir scores” firmly in the “not yet” drawer. Today I have the first twelve bars of a scribbled-over score as a clean transcription, including all the handwritten changes; a piano reduction for rehearsals; and (with a few clicks in MuseScore) practice tracks for every voice.

It is not a one-click solution yet, and I would not trust the result without a quick look through it; review still matters, just as it does with generated code. But I am quite sure that this will improve further (possibly in surprising ways, again).

What we should take from this as developers:

I feel like thinking outside the box is especially worth it today, as the boundaries we have long accepted might have moved quite a bit.