Abstract
BACKGROUND/PURPOSE: ChatGPT, a large language model, can provide an instant and personalized solution in a conversational format. Our study aimed to assess the potential application of ChatGPT-4, ChatGPT-4o without a prompt (ChatGPT-4o-P(-)), and ChatGPT-4o with a prompt (ChatGPT-4o-P(+)) in helping dental students to study oral pathology (OP) by evaluating their performance in answering the OP multiple choice questions (MCQs) of various types or subjects. MATERIALS AND METHODS: A total of 280 OP MCQs were collected from Taiwan National Dental Licensing Examinations. The chatbots of ChatGPT-4, ChatGPT-4o-P(-), and ChatGPT-4o-P(+) were instructed to answer the OP MCQs of various types and subjects. RESULTS: ChatGPT-4o-P(+) achieved the highest overall accuracy rate (AR) of 90.0 %, slightly outperforming ChatGPT-4o-P(-) (88.6 % AR) and significantly exceeding ChatGPT-4 (79.6 % AR, P < 0.001). There was a significant difference in the AR of odd-one-out questions between ChatGPT-4 (77.2 % AR) and ChatGPT-4o-P(-) (91.3 % AR, P = 0.015) or ChatGPT-4o-P(+) (92.4 % AR, P = 0.008). However, there was no significant difference in the AR among three different models when answering the image-based and case-based questions. Of the 11 different OP subjects of single-disease, all three different models achieved a 100 % AR in three subjects; ChatGPT-4o-P(+) outperformed ChatGPT-4 and ChatGPT-4o-P(-) in other 3 subjects; ChatGPT-4o-P(-) was superior to ChatGPT-4 and ChatGPT-4o-P(+) in another 3 subjects; and ChatGPT-4o-P(-) and ChatGPT-4o-P(+) had equal performance and both were better than ChatGPT-4 in the rest of two subjects. CONCLUSION: In overall evaluation, ChatGPT-4o-P(+) has better performance than ChatGPT-4o-P(-) and ChatGPT-4 in answering the OP MCQs.