Enhancing Empathic Communication in Healthcare Education Through Virtual Conversations: Leveraging Large Language Models for Real-Time Feedback

Virtual conversations are increasingly utilized in healthcare education to enhance verbal empathic communication skills through tailored feedback on trainees' responses. These conversations, supported by modalities such as speech, animation, and gestures, are highly customizable and accessible via the internet, bypassing the need for head-mounted displays (HMDs). However, training models to accurately evaluate empathic responses and generate human-like language with actionable suggestions remains a challenge. The advent of large language models (LLMs) provides new solutions to these challenges. This paper examines the impact of GPT-4-generated feedback on the empathic expressions of health professions trainees during virtual conversations. We enrolled 72 students from nursing and dental disciplines in an Interprofessional Collaborative Care course at the University of Florida. Participants were randomly assigned to one of two groups: one received feedback and suggestions from GPT-4 during conversations, while the other did not. We collected data on the perceived accuracy and helpfulness of the feedback from the intervention group. Using the Empathic Communication Coding System (ECCS) and GPT-4 Turbo, we assessed and compared the empathy levels of participants between the two groups. A Mann-Whitney U test was used to determine differences in average empathy levels. Results showed that participants receiving GPT-4 feedback had significantly higher median empathy levels than those without feedback. Feedback's accuracy and utility were also affirmed by the participants. This study highlights the effectiveness of integrating LLMs like GPT-4 into virtual conversations for enhancing training outcomes in healthcare education.