Abstract
To compare the diagnostic accuracy of an artificial intelligence chatbot and clinical experts in vertigo-related diseases and evaluate the ability of the AI chatbot to address vertigo-related issues. 20 clinical questions about vertigo were input to ChatGPT-4o, and three otologists evaluated the responses using a 5-point Likert scale for accuracy, comprehensiveness, clarity, practicality, and credibility. Readability was assessed using Flesch Reading Ease and Flesch-Kincaid Grade Level formulas. The model and two otologists diagnosed 15 outpatient vertigo cases, and the diagnostic accuracy was calculated. The Kruskal-Wallis test, Analysis of Variance (ANOVA), and paired t-test were employed for statistical analysis. ChatGPT-4o scored highest in credibility (4.78). Repeated Measures ANOVA showed that ChatGPT's responses to the 20 questions exhibited statistically significant differences across the five scoring dimensions (F = 2.682, p = 0.038). Readability analysis showed that diagnosis-related outputs were more challenging compared to other types of content. The model's diagnostic accuracy was comparable to a clinician with one year of experience but inferior to a clinician with five years of experience, and the differences in accuracy among the three methods are statistically significant (p = 0.04). ChatGPT-4o shows promise as a supplementary tool for managing vertigo but requires improvements in readability and diagnostic capabilities.