Abstract
Background: Gastrointestinal (GI) cancers remain a major global health concern, where timely and accurate interpretation of endoscopic findings plays a decisive role in patient outcomes. In recent years, deep learning-based decision support systems have shown considerable potential in assisting GI diagnosis; however, their broader adoption is often limited by patient privacy regulations, uneven data availability, and the fragmented nature of clinical data across institutions. Federated learning (FL) offers a practical solution by enabling collaborative model training while keeping patient data local to each hospital. Methods: Vision Transformers (ViTs) are particularly well suited for endoscopic image analysis due to their ability to capture long-range contextual information. Nevertheless, their high computational and communication costs pose a significant challenge in federated settings, especially when data distributions vary across clients. To address this issue, we propose a privacy-preserving federated framework that combines ViTs with a Clinical-Entropy Guided Quantum Evolutionary Algorithm (CEQEA) for adaptive token pruning. The CEQEA leverages the diagnostic diversity of each client's local dataset to guide population initialization, evolutionary updates, and mutation strength, allowing the pruning strategy to adapt naturally to different clinical profiles. Results: The proposed framework was evaluated on curated upper- and lower-GI tract subsets of the HyperKVASIR dataset under realistic non-IID federated conditions. On the final test sets, the model achieved a mean micro-averaged accuracy of 92.33% for lower-GI classification and 90.19% for upper-GI classification, while maintaining high specificity across all diagnostic classes. At the same time, the adaptive pruning strategy reduced the number of tokens processed by approximately 40% and decreased the number of required federated communication rounds by 33% compared to ViT-based federated baselines. Conclusions: Overall, these results indicate that entropy-aware, quantum-inspired evolutionary optimization can effectively balance diagnostic performance and efficiency, making transformer-based models more practical for privacy-preserving, multi-institutional gastrointestinal endoscopy.