Abstract
BACKGROUND: Evidence that artificial intelligence (AI) may improve melanoma detection has led to calls for increased human-AI collaboration in clinical workflows. However, AI-based support may entail a wide range of specific functions for AI. To appropriately integrate AI into decision-making processes, it is crucial to understand the precise role that clinicians see AI playing within their clinical deliberations. OBJECTIVE: This study aims to provide an in-depth understanding of how a range of clinicians involved in melanoma screening and diagnosis conceptualize the role of AI within their decision-making and what these conceptualizations mean for good decision-making. METHODS: This qualitative exploration used in-depth individual interviews with 30 clinicians, predominantly from Australia and New Zealand (n=26, 87%), who engaged in melanoma detection (n=17, 57% dermatologists; n=6, 20% general practitioners with an interest in skin cancer; and n=7, 23% melanographers). The vast majority of the sample (n=25, 83%) had interacted with or used 2D or 3D skin imaging technologies with AI tools for screening or diagnosis of melanoma, either as part of testing through clinical AI reader studies or within their clinical work. RESULTS: We constructed the following 5 themes to describe how participants conceptualized the role of AI within decision-making when it comes to melanoma detection: theme 1 (integrative theme)-the importance of good clinical judgment; theme 2-AI as just one tool among many; theme 3-AI as an adjunct after a clinician's decision; theme 4-AI as a second opinion for unresolved decisions; theme 5-AI as an expert guide before decision-making. Participants articulated a major conundrum-AI may benefit inexperienced clinicians when conceptualized as an "expert guide," but overreliance, deskilling, and a failure to recognize AI errors may mean only experienced clinicians should use AI "as a tool." However, experienced clinicians typically relied on their own clinical judgment, and some could be wary of allowing AI to "influence" their deliberations. The benefit of AI was often to reassure decisions once they had been reached by conceptualizing AI as a kind of "checker," "validator," or in a small number of equivocal cases, as a genuine "second opinion." This raised questions about the extent to which experienced clinicians truly seek to "collaborate" with AI or use it to inform decisions. CONCLUSIONS: Clinicians conceptualized AI support in an array of disparate ways that have implications for how AI should be incorporated into clinical workflows. A priority for clinicians is the conservation of good clinical acumen, and our study encourages a more focused engagement with users about the precise way to incorporate AI into the clinical decision-making process for melanoma detection.