Abstract
Background: Malignant pleural effusions (MPEs) are common in advanced lung cancer patients. Cytological examination of pleural fluid is essential for identifying cell types but presents diagnostic challenges, particularly when reactive mesothelial cells mimic neoplastic cells. AI-powered diagnostic systems have emerged as valuable tools in digital cytopathology. This study explores the applicability of machine-learning (ML) models and highlights the importance of accessible tools for clinicians, enabling them to develop AI solutions and make advanced diagnostic tools available even in resource-limited settings. The focus is on differentiating normal/reactive cells from neoplastic cells in pleural effusions linked to lung adenocarcinoma. Methods: A dataset from the Cytopathology Unit at the Sant'Andrea University Hospital comprising 969 raw images, annotated with 3130 single mesothelial cells and 3260 adenocarcinoma cells, was categorized into two classes based on morphological features. Object-detection models were developed using YOLOv8 and the latest YOLOv11 instance segmentation models. Results: The models achieved an Intersection over Union (IoU) score of 0.72, demonstrating robust performance in class prediction for both categories, with YOLOv11 showing performance improvements over YOLOv8 in different metrics. Conclusions: The application of machine learning in cytopathology offers clinicians valuable support in differential diagnosis while also expanding their ability to engage with AI tools and methodologies. The diagnosis of MPEs is marked by substantial morphological and technical variability, underscoring the need for high-quality datasets and advanced deep-learning models. These technologies have the potential to enhance data interpretation and support more effective clinical treatment strategies in the era of precision medicine.