Abstract
United States federal agencies evaluate healthcare providers to identify, flag, and potentially penalize those that deliver low-quality care compared to national expectations. In practice, evaluation metrics are inevitably impacted by unobserved confounding factors, which reduce flagging accuracy and cause the statistics to be overdispersed relative to the theoretical null distributions. In response to this issue, several authors have proposed individualized empirical null (IEN) methods to estimate an appropriate null distribution for each provider's evaluation statistic while taking into account the provider's effective size. However, existing IEN methods require that the statistics asymptotically follow normal distributions, which often does not hold in applications with small providers or misspecified models. In this article, we develop an IEN framework for exact hypothesis tests that accounts for the impact of unobserved confounding without making any asymptotic assumptions. Simulations show that the proposed IEN method has greater flagging accuracy compared to conventional approaches. We apply these methods to evaluate dialysis facilities and transplant centers that are monitored by the Centers for Medicare and Medicaid Services.