An AAC Application for Generating Japanese Response Phrases Using GPT-4

As augmentative and alternative communication (AAC) users increasingly share their anecdotes, experiences, and jokes, communication with them can become more interactive. We developed a new AAC application that uses speech recognition and the Generative Pre-trained Transformer 4 (GPT-4) to generate four types of Japanese response phrases: boke, tsukkomi, neutral, and backchannel phrases, where boke and tsukkomi phrases are known as Japanese jokes often used in "manzai." A 22-year-old Japanese man with cerebral palsy participated in the development as an AAC user. We conducted a demonstration case study of four sessions with him, each lasting 20 to 30 min. During these sessions, our application was used in peer-to-peer conversations with coevals and opinions for further improvements were solicited. The results were summarized as follows: (1) Our application could generate four types of Japanese phrases in a mean duration of 5.37 s (SD 2.47) after the partner's utterance; (2) The participant selected one tsukkomi, two boke, and four neutral phrases. The mean duration to produce the selected utterance from the end of the partner's utterance was 17.52 s (SD 14.71); (3) When the participant presented the generated Japanese jokes, the participant laughed and nodded at the punctuation pause or at the end of the speech output.