Abstract
BACKGROUND: Accurate understanding of 3D medical images requires a background of specialized medical knowledge. There is a pressing need for easy-to-understand medical visualization tools to help patients accurately interpret 3D image data, especially given the large number of patients requiring such assistance. OBJECTIVE: In this paper, we explore the design considerations of a multimodal medical visualization tool for interpreting 3D medical images, which can help users to understand and recognize 3D medical image data. METHODS: An observational study and focus group interviews were conducted to explore how patients interact with physicians and the main problems they encounter when interpreting 3D medical images. Additionally, we conducted semi-structured expert interviews with physicians to investigate the common methods, techniques, and challenges involved in doctor-patient communication when interpreting 3D medical images. We also organized a participatory design workshop to discuss the patients' design preferences for medical visualization tools. RESULTS: The study identified three types of physician-patient interactions, eight specific behaviors, and seven main issues. It also summarized eight common methods and techniques to aid in understanding 3D medical images and highlighted five key findings regarding design preferences for medical visualization tools. Based on previous studies and our empirical research results, we propose seven design considerations for designing visual interfaces, interaction design plans, audios, infographics, and animation guides. The comprehensive summary of the weights for the above-mentioned design consideration was obtained. A comprehensive weighting of design consideration elements was calculated based on the Analytic Hierarchy Process. The results show that the design consideration factors (A primary factors) that have the relatively big weights are the interaction design (57.091%) and visual interface (25.352%), and the ones that have relatively small weights are the medical education and popularization (12.766%), and text presentation (4.791%). Additionally, we found that the weights of factors of the design considerations (B primary factors) are different in the web application, software and VR/AR platforms. Furthermore, we presented a case study of the design of a multimodal medical visualization tool applied in the medical context to help patients interpret 3D medical image data and improve doctor-patient communication skills. CONCLUSION: This study explores the benefits of combining multiple visualization methods for both doctors and patients. We also discuss the advantages and challenges of designing and using multimodal visualization tools in medical settings.