Abstract
OBJECTIVE: Accurate evaluation of thyroid nodules is crucial for effective management; however, methods such as ultrasonography and Fine Needle Aspiration Cytology (FNAC) can be subjective and operator-dependent. Indeterminate thyroid nodules (ITNs) complicate diagnosis, coming at the expense of time, money, and potentially additional FNA samplings, causing more discomfort for the patients. Recent advancements in artificial intelligence (AI) assisted ultrasound diagnosis system have demonstrated excellent diagnostic performance and the potential to aid in the differentiation of ITNs. This study aims to develop an AI classifier that integrates the AI-assisted ultrasound diagnosis system, FNAC, and demographic data to enhance the differentiation of benign and malignant thyroid nodules, and to compare the diagnostic performance of the models, with a focus on diagnosing ITNs. MATERIALS AND METHODS: In the present research, 620 thyroid nodules were collected from a single medical center and divided into training and testing cohorts (Testing1). We developed five AI models using distinct classification algorithms (Logistic Regression, Support Vector Machine, K-Nearest Neighbor, Random Forest, and Gradient Boosting Machine) that integrate demographic data, cytological findings, and an AI-assisted ultrasound diagnostic system for thyroid nodule assessment. These models underwent prospective validation (Testing2, n = 243) to identify the optimal model. A subsequent prospective study (Testing3) involving 70 thyroid nodules further evaluated the model's performance, where the selected optimal model was compared against FNAC combined with BRAF V600E mutation analysis. RESULTS: After validation with the Testing1 and Testing2 cohorts, the Random Forest (RF) model demonstrated the best overall performance among the five classifiers. The area under the curve (AUC) for the RF model to diagnose thyroid nodules was 0.994 in the training cohort, 0.993 in the testing cohort, and 0.977 in the prospective data. In addition, for 42 included ITNs in the prospective data, the accuracy, sensitivity, and specificity of the RF model were 90.48%, 89.47%, and 91.30%, respectively. In the Testing 3 cohort, the RF model demonstrated superior diagnostic performance compared to both the standalone AI ultrasound auxiliary diagnostic system and FNAC alone. Its accuracy was comparable to FNAC combined with BRAF V600E mutation analysis. Conclusion: Our developed thyroid nodule AI diagnostic model shows favorable predictive value. It can serve as a decision support tool for non-thyroid specialists and assist thyroid surgeons in the management of ITN. CONCLUSION: Our developed thyroid nodule AI diagnostic model shows favorable predictive value. It can serve as a decision support tool for non-thyroid specialists and assist thyroid surgeons in the management of ITN.