Abstract
PURPOSE: To evaluate the variability of P-score grading within the same eye during a single imaging session (P-Var) as a function of retinopathy of prematurity (ROP) severity. METHODS: This study analyzed retinal images from 20 consecutive subjects selected from the Stanford University Network for Diagnosis of ROP (SUNDROP) database who were screened between January 2023 and December 2024 and subsequently received treatment. Nine masked graders evaluated the P-scores of 1597 images from 129 imaging sessions. Variability was measured through calculation of range, intergrader reliability, and agreement metrics. Probabilistic modeling was conducted to assess the impact of limited image sampling on disease classification. RESULTS: Significant P-score variability was observed within single eye visits, increasing with worsening vascular disease (maximum eye visit P-score: β = 0.222, P < 0.001) and overall disease severity as assessed with the Telemedicine ROP Severity Score (β = 0.0073, P < 0.001). Intergrader agreement for the nine graders was good (intraclass correlation coefficient [ICC] = 0.68; 95% confidence interval [CI], 0.65-0.71). Mean ± SD weighted kappa was 0.66 ± 0.10 between grader pairs. Exact P-score agreement was 28.5% ± 5.6%, improving to 72.5% ± 8.2% when allowing a ±1 P-score unit difference. The coefficient of variation of image-level P-scores between graders decreased with worsening disease. Probabilistic modeling demonstrated that limited image sampling with fewer than five views per eye can lead to significant misclassification, particularly in plus eyes. CONCLUSIONS: P-score variability exists within the same imaging session and increases with disease severity. These findings suggest that multiple images per session are necessary for accurate disease classification, particularly in eyes with more severe disease.