Abstract
BACKGROUND: Rapid on-site evaluation (ROSE) plays an important role during transbronchial sampling, providing an intraoperative cytopathologic evaluation. However, the shortage of cytopathologists limits its wide application. This study aims to develop a deep learning model to automatically analyze ROSE cytological images. METHODS: The hierarchical multi-label lung cancer subtyping (HMLCS) model that combines whole slide images of ROSE slides and serum biological markers was proposed to discriminate between benign and malignant lesions and recognize different subtypes of lung cancer. A dataset of 811 ROSE slides and paired serum biological markers was retrospectively collected between July 2019 and November 2020, and randomly divided to train, validate, and test the HMLCS model. The area under the curve (AUC) and accuracy were calculated to assess the performance of the model, and Cohen's kappa (κ) was calculated to measure the agreement between the model and the annotation. The HMLCS model was also compared with professional staff. RESULTS: The HMLCS model achieved AUC values of 0.9540 (95% confidence interval [CI]: 0.9257-0.9823) in malignant/benign classification, 0.9126 (95% CI: 0.8756-0.9365) in malignancy subtyping (non-small cell lung cancer [NSCLC], small cell lung cancer [SCLC], or other malignancies), and 0.9297 (95% CI: 0.9026-0.9603) in NSCLC subtyping (lung adenocarcinoma [LUAD], lung squamous cell carcinoma [LUSC], or NSCLC not otherwise specified [NSCLC-NOS]), respectively. In total, the model achieved an AUC of 0.8721 (95% CI: 0.7714-0.9258) and an accuracy of 0.7184 in the six-class classification task (benign, LUAD, LUSC, NSCLC-NOS, SCLC, or other malignancies). In addition, the model demonstrated a κ value of 0.6183 with the annotation, which was comparable to cytopathologists and superior to trained bronchoscopists and technicians. CONCLUSION: The HMLCS model showed promising performance in the multiclassification of lung lesions or intrathoracic lymphadenopathy, with potential application to provide real-time feedback regarding preliminary diagnoses of specimens during transbronchial sampling procedures. CLINICAL TRIAL NUMBER: Not applicable.