Abstract
BACKGROUND: Colorectal cancer (CRC) is one of the most common malignancies globally and a major cause of cancer-related deaths. In the molecular diagnosis of CRC, microsatellite instability (MSI) status and mutations in genes such as BRAF, KRAS, and NRAS are important molecular markers. Traditional molecular detection methods are costly and time-consuming. Therefore, this study proposes a fine-grained classification method for CRC based on hematoxylin and eosin (H&E) stained tissue section images combined with deep learning (DL) technology, aiming to provide new insights into the molecular diagnosis of CRC. METHODS: In this study, we first collected H&E-stained tissue section images of 383 CRC patients from The First Hospital of Lanzhou University (LZUFH) and constructed the LZUFH_CRC dataset. Then, we proposed a hybrid DL model combining Convolutional Neural Network (CNN) and Vision Transformer (ViT) for fine-grained classification tasks in CRC. The model consists of three parts: a feature extractor, an aggregator, and a classification head. A two-stage training strategy was adopted for model training. Finally, we evaluated the performance of the model on the LZUFH_CRC dataset and compared it with other methods. RESULTS: The results showed that the proposed model achieved an overall accuracy (ACC) of 0.524 and area under the receiver operating characteristic curve (AUC) of 0.791 on the LZUFH_CRC dataset. Among them, the grouping names MSI and NRAS had better classification performance, with F1-scores of 0.724 and 0.514, respectively. Additionally, the study visualized the feature activation maps to show the regions of interest of the model for different input images, finding that the model paid more attention to the transitional areas between tumor and non-tumor regions and the mesenchymal areas of the tumor. Meanwhile, comparisons among different clinical characteristic groups showed that the model did not exhibit significant biases in terms of gender, age and tumor location. CONCLUSIONS: This study proposed a fine-grained classification method for CRC based on DL technology, which combines H&E-stained tissue section images with DL technologies such as CNN and ViT, providing new insights into the molecular diagnosis of CRC. Although the performance of the model needs further improvement, the results indicate that DL technology has potential in the molecular detection of CRC. In the future, the research team will continue to optimize the model to improve the ACC and efficiency of fine-grained classification in CRC.