Abstract
Clinical algorithms are commonly used as decision-support tools, incorporating patient-specific characteristics to predict health outcomes. Risk calculators are clinical algorithms particularly suited for resource allocation based on risk estimation. Although these calculators typically use physiologic data in estimation, they frequently include demographic variables such as race, sex, and age as well. In recent years, the inclusion of race as an input variable has been scrutinized for being reductive, serving as a poor proxy for biological differences, and contributing to the inequitable distribution of services. Little attention has been given to other demographic features, such as sex and age, and their potential to produce similar consequences. By applying a framework for understanding sources of harm throughout the machine learning life cycle and presenting case studies, this paper aims to examine sources of potential harms (i.e. representational and allocative harm) associated with including sex and age in clinical decision-making algorithms, particularly risk calculators. In doing so, this paper demonstrates how systematic discrimination, reductive measurement practices, and observed differences in risk estimation between demographic groups contribute to representational and allocative harm caused by including sex and age in clinical algorithms used for resource distribution. This paper ultimately, urges clinicians to scrutinize the practice of including reductive demographic features (i.e. race, binary-coded sex, and chronological age) as proxies for underlying biological mechanisms in their risk estimations as it violates the bioethical principles of justice and nonmaleficence. Practicing clinicians, including nurses, must have an underlying model literacy to address potential biases introduced in algorithm development, validation, and clinical practice.