Multi-Modal Graph Neural Networks for Colposcopy Data Classification and Visualization.

BACKGROUND: Cervical lesion classification is essential for early detection of cervical cancer. While deep learning methods have shown promise, most rely on single-modal data or require extensive manual annotations. This study proposes a novel Graph Neural Network (GNN)-based framework that integrates colposcopy images, segmentation masks, and graph representations for improved lesion classification. METHODS: We developed a fully connected graph-based architecture using GCNConv layers with global mean pooling and optimized it via grid search. A five-fold cross-validation protocol was employed to evaluate performance before (1-100 epochs) and after fine-tuning (101-151 epochs). Performance metrics included macro-average F1-score and validation accuracy. Visualizations were used for model interpretability. RESULTS: The model achieved a macro-average F1-score of 89.4% and validation accuracy of 92.1% before fine-tuning, which improved to 94.56% and 98.98%, respectively, after fine-tuning. LIME-based visual explanations validated models focus on discriminative lesion regions. CONCLUSIONS: This study highlights the potential of graph-based multi-modal learning for cervical lesion analysis. Collaborating with the MNJ Institute of Oncology, the framework shows promise for clinical use.