Abstract
This study aims to evaluate the impact of artificial intelligence (AI) on inter- and intra-rater agreement in mammography interpretation, comparing improvements in reliability between general and breast subspecialty radiologists in a clinical setting. This study was conducted using anonymized digital mammograms from 65 women aged 40-74 years undergoing routine screening. Fourteen radiologists, grouped by experience, assessed images in a multi-reader, multi-case, crossover design with and without AI assistance. Statistical analyses, including Cohen's Kappa and meta-analysis, measured inter- and intra-rater reliability across radiological variables. AI assistance significantly improved agreement with the gold standard for both general and breast subspecialty radiologists. Variables such as BI-RADS breast density and lesion location showed marked improvements, particularly among general radiologists, where Kappa values for BI-RADS breast density rose from 50.01% to 81.38% with AI. Subspecialists demonstrated smaller performance gains, likely due to higher baseline accuracy. AI also enhanced intra-rater reliability and reduced variability across experience levels. These findings support AI's role as a valuable adjunct in breast cancer screening, addressing the shortage of experienced radiologists. Further research in real-world settings is necessary to confirm these results and optimize AI integration.