Abstract
PURPOSE: This study aims to develop an artificial intelligence (AI) model to assist ophthalmologists in distinguishing ocular surface squamous neoplasia (OSSN) from benign ocular surface lesions using ultrasound biomicroscopy (UBM) images. METHODS: Data were retrospectively collected from 139 patients with biopsy-proven conjunctival lesions, including 201 UBM images of benign lesions (e.g.,pterygium, squamous papilloma) and 381 images of OSSN (e.g.,squamous cell carcinoma, conjunctival intraepithelial neoplasia). Patients with conjunctival pigmented lesions, melanoma, lymphoma, or those without a pathological diagnosis were excluded. UBM images were cropped to the anterior segment region and rescaled to a standard size of 300 × 200 pixels. Data augmentation techniques were applied to enhance the diversity of training images. A convolutional neural network was trained and tested using five-fold cross-validation. A heatmap was generated to illustrate the model's decision-making process. The AI model's performance was compared to that of three human experts with varying levels of experience. Additionally, univariate regression analysis was performed to assess the impact of patient-related factors (age, sex, race/ethnicity, lesion location, and side) on model performance. RESULTS: Our AI model achieved an accuracy of 74.3 ± 3.9%, sensitivity of 75.0 ± 8.6%, specificity 73.0 ± 11.5%, precision of 83.3 ± 4.8%, F1 score (i.e., the harmonic mean of precision and recall) of 0.79 ± 0.06,and area under the receiver operating characteristic (AUROC) curve of 0.83 ± 0.03 in detection of OSSN. It significantly outperformed two ocular oncology fellows (p = 0.02 and 0.03, respectively) and demonstrated borderline significance compared to a senior ophthalmologist (p = 0.05). The heatmaps effectively highlighted the lesions, suggesting that echogenicity played a crucial role in the model's predictions. None of the patient-related factors significantly affected model performance (all p > 0.1), supporting its equitable diagnostic capability across diverse patient groups. CONCLUSION: This study demonstrates the feasibility of using AI to differentiate OSSN from benign conjunctival lesions based on UBM images. The heatmap enhances model transparency, and the consistent performance across patient subgroups highlights its potential as a fair and valuable tool for clinical decision-making in ocular surface tumor evaluation.