Evaluation is Key: Providing Appropriate Evaluation Measures for Participatory and User-Centred Design Processes of Healthcare IT

Introduction: The increasing availability of healthcare IT has the potential to improve the integration of health services. Existing projects developing healthcare IT mostly disregard the potential and importance of incorporating user feedback and proper evaluation measures to gain user feedback throughout the development process. We therefore provide methodological guidance for evaluation in a stepwise user-centred design process. Methods: Based on a literature review we propose adequate methods for data collection in each phase of participatory and user-centred healthcare IT development. In order to provide an orientation within the plethora of development processes used in practice, we consolidate a generic blueprint process from the literature review. The applicability of our methodological guidance is shown in three diverse use cases from the field of integrated care. Results: From 14 literature items, we identified common evaluation methods including, among others, interviews, focus groups, and surveys. These methods can be associated to six typical development phases that could be derived from research: State of the Art Research, Requirement Analysis, Conceptual Prototype, Preliminary Prototype, Full Prototype, Full Application. The use cases demonstrate the value of qualitative methods and mixed methods designs. Discussion: Our methodological guidance has proven applicable for designing healthcare IT solutions from scratch - both for patient and professional settings - and to develop a platform for combining existing component-based solutions. In integrated care settings, where a wide range of stakeholders with multiple needs exist, we thus provide methodological guidance on how to involve users in the development process. Conclusion: Our stepwise methodological guidance helps to design and properly evaluate healthcare IT solutions, which meet the user needs and requirements, for integrated care settings.