Abstract
BACKGROUND: Artificial intelligence (AI) and machine learning are transforming the optimization of clinical and patient workflows in healthcare. There is a need for research to specify clinical requirements for AI-enhanced care pathway planning and scheduling systems to improve human-AI interaction in machine learning applications. The aim of this study was to assess content validity and prioritize the most relevant functionalities of an AI-enhanced care pathway planning and scheduling system. METHODS: A prospective content validity assessment was conducted in five university hospitals in three different countries using an electronic survey. The content of the survey was formed from clinical requirements, which were formulated into generic statements of required AI functionalities. The relevancy of each statement was evaluated using a content validity index. In addition, weighted ranking points were calculated to prioritize the most relevant functionalities of an AI-enhanced care pathway planning and scheduling system. RESULTS: A total of 50 responses were received from clinical professionals from three European countries. An item-level content validity index ranged from 0.42 to 0.96. 45% of the generic statements were considered good. The highest ranked functionalities for an AI-enhanced care pathway planning and scheduling system were related to risk assessment, patient profiling, and resources. The highest ranked functionalities for the user interface were related to the explainability of machine learning models. CONCLUSION: This study provided a comprehensive list of functionalities that can be used to design future AI-enhanced solutions and evaluate the designed solutions against requirements. The relevance of statements concerning the AI functionalities were considered somewhat relevant, which might be due to the low level or organizational readiness for AI in healthcare.