Conversing with business process-aware large language models: the BPLLM framework

Traditionally, process-aware Decision Support Systems (DSSs) have been enhanced with AI functionalities to facilitate quick and informed decision-making. In this context, AI-Augmented Business Process Management Systems have emerged as innovative human-centric information systems, blending flexibility, autonomy, and conversational capability. Large Language Models (LLMs) have significantly boosted such systems, showcasing remarkable natural language processing capabilities across various tasks. Despite the potential of LLMs to support human decisions in business contexts, empirical validations of their effectiveness for process-aware decision support are scarce in the literature. In this paper, we propose the Business Process Large Language Model (BPLLM) framework, a novel approach for enacting actionable conversations with human workers. BPLLM couples Retrieval-Augmented Generation with fine-tuning, to enrich process-specific knowledge. Additionally, a process-aware chunking approach is incorporated to enhance the BPLLM pipeline. We evaluated the approach in various experimental scenarios to assess its ability to generate accurate and contextually relevant answers to users' questions. The empirical study shows the promising performance of the framework in identifying the presence of particular activities and sequence flows within the considered process model, offering insights into its potential for enhancing process-aware DSSs.