Educating Dora: Teaching a conversational agent to talk

Many organisations are witnessing the introduction of automated systems to mediate between them and their clients, often designed as dialogic user interfaces which are conversational in nature. Despite calls for the design of AI-powered conversational agents to draw on Ethnomethodological Conversation Analysis (EMCA), how an EMCA-informed conversation design can improve interactions between humans and conversational agents is yet to be empirically examined. This paper reports on a collaboration between EMCA researchers and AI conversation designers at digital health company Ufonia to develop Dora, a pre-existing 'automated clinical assistant' that provides telephone consultations for patients of the UK's National Health Service. Our analysis identified differences between the conversation design of the product and conversational practices found in equivalent activities involving human clinicians, relevant to implementing changes to improve user experience. We demonstrate that users are prompted to recalibrate their practices (such as for turn-taking) in situ as they engage with a system which does not match their prior experiences of this activity type. Such insights can be leveraged to adapt system design so that it more closely approximates users' prior experiences of such engagements.