Towards Inclusive Education: Multimodal Classification of Textbook Images for Accessibility

To foster inclusive education, accessible educational materials must be tailored to meet the diverse needs of students, especially students with disabilities. Images in educational materials play an important role in understanding textbook exercises but can also distract such students and make them more challenging. Therefore, we need an adaptive system that identifies non-essential images to reduce cognitive load and distractions for students with disabilities. Accordingly, this work proposes a computational framework to categorize textbook exercise images, facilitating their inclusion in accessible textbooks and enhancing learning for visually impaired and neurodevelopmental disorders students. Using three French textbook exercise datasets of 652 (text, image) pairs, we compared monomodal (text-only) and multimodal (text and image) classification approaches. We found that text-based models, particularly CamemBERT, excel in classifying images, achieving an accuracy rate of 85.25% on French text data. We also use Local Interpretable Modelagnostic Explanation for model interpretability and conduct qualitative analyses to deepen insights into model performance. This work is only a very first step towards an automatic translation of inclusive textbooks. Moreover, this paper revealed that this first step is already very challenging. We hope to draw more researchers' attention to this problem.