Recognizing ancient handwritten Math Expression with Encoder-Decodeur approaches

Niveau : Master 2

Période : spring/summer 2025

LS2N / IPI – SPHERE research teams

This internship is a collaborative effort between the IPI (Image Perception Interaction) team at LS2N in Nantes and the SPHERE (Sciences, Philosophy, History) team at Université Paris Cité. It is an interdisciplinary project aimed at assisting the transcription of Leibniz’s manuscripts.

Context

Leibniz’s manuscripts are a collection of essential documents for understanding his thought process and the development of his theories. However, manually transcribing these manuscripts is time-consuming and labor-intensive. To facilitate the work of transcribers, it is crucial to develop automatic transcription techniques. These efforts are often hindered by the complexity of the mathematical notations used. Unique layouts, idiosyncratic notations, and the degradation of archival materials make these documents challenging for state-of-the-art tools to process automatically.

Several approaches exist in the state of the art for handwritten mathematical expression (HME) recognition. This internship will focus initially on encoder-decoder (image-to-sequence) methods, implementing the entire learning pipeline—from defining the training data to evaluating the tools.

This project provides the unique opportunity to combine state-of-the-art deep learning techniques with historical research, making a tangible contribution to preserving and understanding the legacy of one of history’s greatest thinkers.

During this internship, the candidate will work with the state-of-the-art deep learning framework PyTorch, gain expertise in encoder-decoder architectures, and learn to address real-world challenges in document analysis.

Aims / Work

The main goals of this internship are as follows:

• Conduct a comprehensive review of existing open-source solutions for encoder-decoder HME recognition.

• Define the training protocol

• Evaluate the selected approach(es) and propose improvements.

The results of this internship have the potential to lead to contributions in high-impact conferences (e.g., ICDAR) and to the development of open-source tools for historical document analysis.

Contacts : harold.mouchere@ls2n.fr ; davidrabouin2@gmail.com