Comparing the effectiveness of LLM-Powered coaching with human coaching and GPT conversation

This study examines the effectiveness of LLM-powered coaching compared to traditional human coaching and GPT-4 interactions across 11 key criteria of coaching effectiveness. Leveraging a randomized controlled trial with 32 participants, the research revealed LLM-powered coaching significantly outperforms GPT-4 in empathy, engagement, and encouragement - achieving outcomes comparable to human coaching in all dimensions. Notably, LLM-powered coaching showed the greatest improvements in Positive Affect ( + 8.49%) and reductions in Negative Affect ( - 27.45%) among the groups. Qualitative feedback highlights the strengths of human coaching in emotional depth and nonverbal cues, while AI coaching excels in its accessibility, flexibility, and structured support for self-reflection and action planning. This research underscores the transformative potential of AI-powered coaching to foster individuals' growth and well-being on a large scale. By offering accessible, scalable, cost-effective coaching solutions, AI coaching complements traditional human coaching, expanding access to personal and professional development for a broader and more diverse population.