Abstract
Ensuring data privacy in medical image classification is a critical challenge in healthcare, especially with the increasing reliance on AI-driven diagnostics. In fact, over 30% of healthcare organizations globally have experienced a data breach in the last year, highlighting the need for secure solutions. This study investigates the integration of transfer learning and federated learning for privacy-preserving medical image classification using GoogLeNet and VGG16 as baseline models to evaluate the generalizability of the proposed framework. Pre-trained on ImageNet and fine-tuned on three specialized medical datasets for TB chest X-rays, brain tumor MRI scans, and diabetic retinopathy images, these models achieved high classification accuracy across various aggregation methods. Additionally, the proposed dynamic aggregation method was further analyzed using modern architectures, EfficientNetV2 and ResNet-RS, to assess the scalability and robustness of the model. A key contribution is the introduction of a novel adaptive aggregation method, which dynamically alternates between Federated Averaging (FedAvg) and Federated Stochastic Gradient Descent (FedSGD), based on data divergence during communication rounds. This approach optimizes model convergence while preserving privacy in collaborative settings. The results demonstrate that transfer learning, when combined with federated learning, offers a scalable, robust, and secure solution for real-world medical diagnostics, enabling healthcare institutions to train highly accurate models without compromising sensitive patient data.