Abstract
OBJECTIVE: This study aims to evaluate the performance of ChatGPT-4o and DeepSeek-R1 in perioperative medication therapy management for gastric cancer, assessing their reliability and practicality as auxiliary tools in clinical pharmacy education. METHODS: This study utilized a retrospective design to collate issues pertaining to perioperative medication management in gastric cancer, from which a standardized question set was developed. The set was concurrently submitted to both ChatGPT-4o and DeepSeek-R1 to generate model responses. Two independent assessors, blinded to the model sources, evaluated the outputs according to a predefined framework covering three core domains: (1) Clinical applicability, assessed via a 7-point Likert scale; (2) Information quality, evaluated using the DISCERN instrument for evidence reliability and content completeness; and (3) Readability, measured through the Flesch Reading Ease Score (FRES) and the SMOG Index. RESULTS: In the 24-item evaluation of perioperative drug therapy for gastric cancer, both models exhibited high inter-rater reliability, with Cronbach's α values of 0.880 for DeepSeek-R1 and 0.852 for ChatGPT-4o. DeepSeek-R1 demonstrated superior performance in clinical applicability (Likert score: 5.63 ± 0.94 vs. 5.10 ± 0.78, p < 0.001) and information quality (DISCERN score: 54.50 ± 6.71 vs. 50.56 ± 6.08), although neither model reached the excellence threshold (≥65 points). Readability assessment revealed moderately complex text difficulty, with Flesch Reading Ease scores below 30 and SMOG indices indicating a reading level of ≥17 years, which remains appropriate for undergraduate clinical pharmacy education. CONCLUSION: Both ChatGPT-4o and DeepSeek-R1 have demonstrated potential in addressing issues related to perioperative medication management for gastric cancer, with their generated responses showing good practical Applicability and readability suitable for the clinical pharmacy professional community. However, it should be noted that the quality of information provided by both models does not currently meet professional standards for drug therapy management. Therefore, they can be utilized as auxiliary tools for training the analytical skills of undergraduate students in clinical pharmacy, but their use should be guided by mentors.