Development of a generative AI-powered teachable agent for middle school mathematics learning: A design-based research study

This paper reports on a design-based research (DBR) study that aims to devise an artificial intelligence (AI)-powered teachable agent that supports secondary school students' learning-by-teaching practices of mathematics learning content. A long-standing pedagogical practice of learning-by-teaching is powered by a recent advancement of generative AI technologies, yielding our teachable agent called ALTER-Math. This study chronicles one usability testing and three cycles of iterative design and implementation process of ALTER-Math. The three empirical studies involved a total of 320 middle school students and six teachers in authentic classroom settings. The first study was exploratory, focusing on the qualitative feedback from the students and teachers through open-ended surveys, interviews and classroom observations. The second study yielded a medium-high (M = 3.26) quantitative survey result on students' perceived engagement and usability on top of the qualitative findings. Finally, the final study included pre- and post-knowledge tests in a quasi-experimental study design as well as student and teacher interviews. The final study revealed a bigger significant knowledge improvement in students who used ALTER-Math compared to the control group, suggesting a positive impact of AI-powered teachable agents on students' learning. The design implications learned from multiple iterations are discussed to inform the future design of AI-powered learning technologies.Practitioner notes What is already known about this topic Learning-by-teaching is a long-standing effective pedagogical strategy to enhance students' domain knowledge and feelings of responsibility in learning. Various teachable agents have been developed and have demonstrated benefits in students' learning. Generative AI offers the potential to provide naturalistic, contextualised and adaptive conversations. What this paper adds Develops a novel generative AI-powered teachable agent for middle school mathematics learning, called ALTER-Math. Reports the iterative design process involving empirical classroom implementations of ALTER-Math. Reveals a bigger significant improvement in the student's mathematical knowledge after using ALTER-Math, compared to the control group. Implications for practice and/or policy Researchers can be inspired by this design example of a theoretically grounded generative AI learning technology. Educational technology designers could hear the real voices of students and teachers about the generative AI learning technologies. Researchers and educational technology designers could be directed by the design implications to the future design of AI-powered learning technologies and teachable agents.