Abstract
The concept of alignment has undergone a remarkable rise in recent years to take center stage in the ethics of artificial intelligence. There are now numerous philosophical studies of the values that should be used in this ethical framework as well as a technical literature operationalizing these values in machine learning models. This article takes a step back to address a more basic set of critical questions: Where has the ethical imperative of alignment come from? What is the ethical logic of alignment-how, exactly, does it propose to regulate machines' and peoples' conduct? And what are the social and political implications of this ethics? After discussing the logical and normative implications of the term itself-in what sense alignment can have an ethical meaning-we undertake a four-part "anatomy" of alignment in contemporary large language models (LLMs): first, a relatively technical sense in sequence modeling; second, a more normative sense relating to how outputs of pre-trained models are ethically evaluated; a third sense where external values are introduced using fine-tuning techniques to manage undesired model behaviors; and fourth sense, where alignment is given extreme ethical stakes in philosophical discussions of existential risks. We find that the ethics of alignment is fundamentally concerned with the problem of control, with unintended model behaviors that arise from divergences between training objectives and the normative expectations that govern the contexts in which they are used. Alignment serves to bridge the gap between what we call an "is" normativity, of statistical patterns identified by models and an "ought" normativity where values are technically introduced in models to steer them away from undesired behaviors. By problematizing control, the ethics of alignment weakens capacities to both make more substantive ethical judgments and also political decisions about how to live with AI.