Abstract
BACKGROUND: Immune checkpoint inhibitor (ICI) therapy has significantly improved treatment outcomes for various cancers by enhancing T cell-mediated anti-tumor immune responses. However, accurately predicting patient response to ICI treatment remains a major challenge due to the risk of immune-related adverse events. Microsatellite instability (MSI), as an important molecular biomarker characterized by high mutation rates and abundant tumor neoantigen production, has been demonstrated to effectively predict clinical benefits from immunotherapy. In gastric adenocarcinoma (STAD) patients, approximately 22% exhibit the MSI subtype while the majority are microsatellite stable (MSS). This significant molecular heterogeneity underscores the urgent need to develop reliable predictive tools. METHODS: To address this problem, we developed a multimodal deep learning model named ProMMF_Kron based on a multicenter dataset comprising 282 patients. The model employs a two stage feature fusion strategy: first extracting key features from both molecular profiles and pathological images through differential gene analysis and a pretrained deep convolutional neural network, respectively; then designing a sophisticated fusion architecture incorporating Kronecker product operations and back-projection modules to achieve efficient interaction between gene expression features and pathological image features. The dataset was partitioned into training, validation, and testing sets at a ratio of 6:2:2. RESULTS: Experimental results demonstrate that the ProMMF_Kron model effectively distinguishes between MSI and MSS subtypes (MSI versus MSS) and exhibits competitive predictive performance on independent test datasets, achieving an AUC of 0.96 (95% CI: 0.89-1.00), outperforming traditional single-modality prediction models (3.2% AUC improvement) and other multimodal fusion approaches (4.3% AUC improvement). Further validation confirms the model's excellent stability and generalization capability, maintaining high predictive accuracy on colorectal cancer (CRC) dataset. DISCUSSION: Through bioinformatics analysis and feature visualization techniques, this study also reveals potential associations between key molecular biomarkers and critical immune regulatory pathways, providing a powerful decision-support tool for precision immunotherapy in gastric cancer with substantial clinical translation value and application prospects.