Conclusions
Our results highlight the importance of validation, avoiding conclusions based on P value thresholds, and providing comparable metrics. Further transparency of data and equal weighting of negative and positive findings in explorations of novel biomarkers will allow investigators to more quickly weed out less promising biomarkers.
Objective
Biomarker studies are important for generating mechanistic insight and providing clinically useful predictors of chronic kidney disease (CKD) progression. However, variability across studies can often muddy the evidence waters. Here we evaluated real-world variability in biomarker studies using two published studies, independently conducted, of the novel plasma marker soluble urokinase-type plasminogen activator receptor (suPAR) for predicting CKD progression in children with CKD. Study design: A comparison of 2 prospective cohort studies. Setting & participants: 541 children from the Chronic Kidney Disease in Children (CKiD) study, median age 12 years, median glomerular filtration rate (GFR) of 54 mL/min/1.73m2. Outcome: The first occurrence of either a 50% decline in GFR from baseline or incident end-stage kidney disease. Analytical approach: The suPAR plasma marker was measured using the Quantikine ELISA immunoassay in the first study and Meso Scale Discovery (MSD) platform in the second. The analytical approaches varied. We used suPAR data from the 2 assays and mimicked each analytical approach in an overlapping subset.
Results
We found that switching assays had the greatest impact on inferences, resulting in a 38% to 66% change in the magnitude of the effect estimates. Covariate and modeling choices resulted in an additional 8% to 40% variability in the effect estimate. The cumulative variability led to different inferences despite using a similar sample of CKiD participants and addressing the same question. Limitations: The estimated variability does not represent optimal repeatability but instead illustrates real-world variability that may be present in the CKD biomarker literature. Conclusions: Our results highlight the importance of validation, avoiding conclusions based on P value thresholds, and providing comparable metrics. Further transparency of data and equal weighting of negative and positive findings in explorations of novel biomarkers will allow investigators to more quickly weed out less promising biomarkers.