Abstract
One natural motivation for designing artificial intelligence (AI) with ethical capacities is to mitigate the risk that powerful AI systems will harm us. Against this idea, some authors have argued we do not need ethical AI in order to prevent harm to humans. We simply need safe AI. In this paper, I consider an argument of this type and raise objections to it. In particular, I argue that the goal of implementing safety features in AI systems is a much more complicated task than such authors have acknowledged, and moreover I maintain that merely safe AI could be ethically problematic in numerous ways. Then, I show that a certain kind of ethical AI, which I call end-autonomous ethical AI, would be especially dangerous since their autonomous capacities would make possible numerous risks. Finally, I motivate the case for a specific category of ethical AI, namely, end-constrained ethical AI. I describe these systems as possessing whatever capacities would be necessary for satisfying ethical aims beyond safety while lacking end-autonomy. In short, the goal of this paper is to establish that end-constrained ethical AI occupy a desirable middle ground between merely safe AI and end-autonomous ethical AI because they allow us to secure more ethical goods than just safety, and they are also not as risky as end-autonomous ethical AI.