The research focuses on the use of artificial intelligence methods for the automatic recognition and transcription of historical musical notation which, due to its complexity, has for decades remained beyond the reach of digital tools. Should the ongoing pilot studies yield positive results, a breakthrough in the digital transcription of lute music preserved in manuscripts may become possible. Eliseo Fuentes-Martínez will remain in Wrocław until mid-February 2026.

We invite readers to a discussion on how algorithms can assist in deciphering music notated several centuries ago, and whether algorithms are capable of learning to understand historical systems of musical notation that pose challenges even for experienced musicologists.

Ewelina Kośmider: Could you explain, in general terms, what optical music recognition is?

Eliseo Fuentes-Martínez: Optical music recognition, or OMR, is a field within artificial intelligence that uses computer vision to analyse musical sources such as printed scores and handwritten manuscripts. The system processes images of complete manuscripts or selected fragments and automatically transcribes the musical notation into a digital format.

Marianna Siatkowska: To make OMR easier to understand, it helps to compare it with OCR, optical character recognition. Today, we can photograph a printed or even handwritten page and instantly extract editable text. With music, especially historical music, this process is still at an early stage. For standard modern notation, results are improving, but for other systems, such as tablatures or mensural notation, the challenges are far greater. The key issue is scale. OMR allows us to analyse large repertoires rather than isolated pieces. Although lute tablature may seem like a narrow field, it is crucial, as it was long neglected in musicology despite the importance of the music preserved in this notation.

GJ: I would like to add two concrete examples. First, consider international catalogues such as RISM, which usually present only musical incipits. These fragments tell us very little about how a piece develops. Second, much lute music survived exclusively in tablature. For decades, these sources remained marginalised because of their notation. With OMR and encoding standards such as MEI (Music Encoding Initiative), we can translate these sources into formats that are readable both by scholars and by performers unfamiliar with tablature. This makes an enormous body of music accessible again it also allows you to store the content of the composition.

EK: So, if I understand correctly, these manuscripts are difficult for modern performers to read or play directly?

E F-M: Not exactly. For trained musicians and musicologists, these sources are readable. However, an AI model has no prior knowledge of musical notation. It must be taught how to interpret what it sees, which is precisely why this research is necessary.

Maria Kozan: Why is musicology so closely connected with artificial intelligence, and with OMR in particular?

Grzegorz Joachimiak: This question is closely linked to the establishment of the Centre for Digital Musicology at our university. After many years of working with historical sources, I realised that traditional, manual methods are no longer sufficient. We are dealing with an enormous quantity of material. With the support of machines, AI models, and Large  Language Model (LLM) systems, we can analyse data on a scale that would otherwise take far longer than a human lifetime. The aim is not to replace musicologists, but to teach models to recognise what we ourselves recognise: what is crucial, what is secondary, and what truly matters in a source. This collaboration between musicology and information technology is essential.

At the University of Alicante, for example, there is no separate department of musicology, but rather a strong integration within an IT-focused environment.

E F-M: We are part of the Department of Languages and Systems, which belongs to the broader IT structure. Within it, we work in the Pattern Recognition and Artificial Intelligence Group (PrP), one branch of which specialises in optical music recognition

.

GJ: It is worth stressing that the use of AI (including machine learning approaches) is not new in science. What is relatively new is their systematic application in musicology, particularly in some countries. In Poland, such work is still rare at universities. Our Centre is currently unique in this respect. We have chosen research cases that are especially important to us, such as lute tablatures from the seventeenth and eighteenth centuries. These notations differ significantly from earlier and later systems and pose major challenges. Similar work has been undertaken by colleagues in Vienna (who focus on the period up to c. 1550), and together we aim to address areas that have not yet been fully explored. One of the most difficult problems is recognising handwritten lute tablature and interpretation of rhythm. We believe we have promising ideas, and Eliso’s contribution is  invaluable.

EK: So you are effectively translating historical notation into modern staff notation?

GJ: Exactly. This is essential because if we ignore sources written in tablature, we risk losing a significant part of our musical heritage. OMR enables us to work not only with incipits, but with complete works. Our colleagues in Vienna focus on so-called German lute tablature notation, which has its own symbols, conventions, and historical layers. Each notation system requires specifically adapted tools and models.

EK: Were there significant differences in musical notation between countries?

GJ: Yes, although these systems functioned more like stylistic traditions than strictly national ones. We speak of Italian, French, German, or Spanish lute tablature, even though they circulated widely across Europe. In Poland, for instance, we know of a Polish lute tablature tradition, although only indirect copies survive from the 17^th century sources (A. Poliski’s collection). In Silesia, we have even a manuscript from 1538-1544 century in which are combine Italian and German lute tablature notations with repertoire which is earlier than date’s indicate. A good example is Wojciech Długoraj (1557/8 – after 1619), a Polish lutenist whose works survived mainly in sources written in German lute tablature notation, like in the manuscript preserved  in Leipzig Stadtbibliothek.

MS: There are vast numbers of tablature sources, far more than we can process at once. Our project therefore focuses on letter tablatures, often referred to as French tablature. We are developing tools that will allow these sources to be easily and effectively transcribed and read more widely.

GJ: Another important aspect concerns music editions. Traditionally, preparing a critical edition could take decades. Digital libraries have already accelerated this process, and OMR allows us to go further beyond transcription, towards large-scale analysis and new research questions. This is where we see our future work, in close cooperation with the University of Alicante, which is a global leader in OMR research, particularly in medieval and mensural notation.

EK: That leads directly to my next question: how did the University of Alicante become such an important centre for OMR research?

MK: And why did you decide to collaborate specifically with Poland, the University of Wrocław?

E F-M: Our group at University of Alicante has been working intensively on OMR for many years, including projects on Gregorian chant and mensural notation. Collaboration with Polish colleagues began earlier this year, when Grzegorz and Marianna visited Alicante. We quickly realised that lute tablature represents both a major challenge and a significant research opportunity. Music notation varies enormously across periods and genres, and at present no single model can handle all of them. Lute tablature therefore deserves focused study.

I would also like to emphasise that one of the core goals of OMR is preservation. In the past, music survived only through handwritten transmission, and many sources have deteriorated or disappeared. Today, we can not only digitise manuscripts, but also preserve structured information about them, making future interpretation far more reliable.

GJ: This shared vision is the foundation of our cooperation. Our expertise in historical lute sources complements the technological strengths of our partners in Alicante. This collaboration emerged from my doctoral research and Marianna’s PhD project. After extensive discussion with colleagues such as Jorge Calvo Zaragoza and David Rizo, we decided to test our ideas through a pilot project. Eliseo’s visit to Wrocław is part of this experimental phase.

MK: Is this project connected to your doctoral research?

E F-M: Yes. My PhD focuses on OMR, AI, and music, particularly multimodality, the integration of images, audio, and encoded notation. Lute tablature is especially suitable for this approach, as its interpretation cannot rely on the manuscript alone.

MK: What methodological challenges do seventeenth- and eighteenth-century handwritten lute tablatures pose for current OMR systems?

E F-M: The greatest challenge is graphical variability. Printed notation is relatively consistent, but handwritten sources vary enormously, even when produced by the same scribe. Differences in angle, spacing, and execution make recognition extremely difficult. Overcoming these discrepancies is one of the central research problems in manuscript OMR.

MS: By training models on many varied examples of the same symbol, we enable them to generalise and learn autonomously. This requires significant effort now, but it will ultimately make research much more efficient.

E F-M: It is a symbiotic process. Musicologists provide annotated data and explain the rules of the notation. The model learns from this expertise and then assists scholars in further annotation and analysis.

MS: Every notation system has its own internal logic. Teaching a model is rather like teaching a child: step by step, we show it what to prioritise and how to interpret symbols within a given context.

GJ: For our pilot study, we selected a single, exceptionally large collection: the Grüssau (Krzeszów)  Lute Music Collection from Silesia. It contains over 2,000 pieces and is one of the largest collections of its kind worldwide. Comparable collections exist only in a few places, such as Benedictine Abbey in Kremsmünster (Austria). This scale makes it ideal for testing whether our methods can be expanded to large datasets.

One additional difficulty lies in interpreting rhythmic signs  in this notation. Recognising symbols is only the first step; understanding their musical meaning is equally complex. We believe we have promising solutions, but this will require further testing.

This project also has broader significance for Poland. Apart from a few specialised institutions, digital musicology, especially for historical sources, is still underdeveloped at Polish universities. Our aim is to change this. In May next year, we will host a major digital musicology meeting in cooperation with the University of Alicante and other partners.

Internationally, only a small number of centres focus on lute tablature. This year, we received a pilot grant within a new Research Consortium under Le Studium Loire Valley Institute for Advanced Studies for 2025. “Tablature and Technology for Tomorrow” will be hosted at the: Centre d’Études Supérieures de la Renaissance (CESR) by Philippe Vendrix under the supervision of Ailin Arjmand (Programme Ricercar) and will bring together an international team of scholars: Kateryna Schöning (University of Vienna), Tim Crawford (Goldsmiths, University of London), John Griffiths (University of Melbourne) and Grzegorz Joachimiak (University of Wrocław). This demonstrates the growing recognition of the field.

MS: Our hope for the future is that the University of Wrocław will become a leading international centre for research on baroque lute tablatures. We have many resources and the potential to make this happen.

MK: What are the long-term implications of this research for accessibility and preservation of early music sources?

E F-M: One of the central aims of OMR is to preserve not only manuscripts as images, but also the knowledge embedded in them. This ensures that future generations can access, analyse, and interpret these sources.

MS: Large, well-structured datasets will allow future scholars to better understand how early music shaped later musical traditions. Ultimately, this work is about safeguarding cultural heritage, not merely applying advanced technology for its own sake.

MK: Could this lead to more performances, concerts, or even new academic programmes devoted to early music?

EK: Do you think your work will make early or Gregorian music more accessible to performers?

E F-M: Absolutely. In a recent project, our team reconstructed a dataset that led directly to a concert performance of music that had not been heard for centuries. Once manuscripts are fully recognised and transcribed, performers can bring this music back to life.

MS: Concerts and recordings are crucial. They connect historical repertoires with contemporary audiences and influence modern musical practice in return.

GJ: I would like to mention another initiative in which Marianna and I are involved.The COST Action EarlyMuse – A New Ecosystem of Early Music Studies, in which I lead the SOURCES  studies working group (WG2). We focus on European musical heritage up to the eighteenth century and beyond, including manuscripts, prints, instruments, and copies. Before we can perform or reinterpret this music, we must first identify and understand what survives. Without technological support, this task would be impossible. AI enables us to confront the vast number of untouched sources still hidden in libraries.

EK: Eliseo, may I ask about your personal background? Do you come from music or IT?

E F-M: Both. I trained as a pianist, studied computer engineering, and then completed a master’s degree in artificial intelligence. My work combines all these areas.

EK: As a non-specialist, I have learned a great deal today. The lute seems central to early music was its repertoire largely forgotten?

GJ: From the sixteenth century onwards, the lute and the organ were among the most important instruments. In the seventeenth and eighteenth centuries, lute music flourished particularly in Central Europe, Silesia, Germany, Austria, and Bohemia. These sources are stylistically rich, combining French influences with local traditions, including devotional and spiritual music. Their handwriting is extremely challenging to analyse, which is why they have been largely neglected. If our research succeeds, it could represent a significant milestone in the study of historical music notation.

Complied by Maria Kozan

Date of publication: 19.12.2025
Added by: M.K.