#7 Machine Ethics and AI Regulation: Some Preliminary Thoughts on Regulating Artificial Moral Agents

This blog post will briefly explore the intersection of two increasingly important areas of concern around artificial intelligence (AI) technologies, namely, machine ethics and AI regulation. While there has been progress in both of these areas in recent years, there is a clear danger that AI regulation might not always keep up with the pace of technological progress in AI. The ongoing research and development of agentic AI systems, which can autonomously perform tasks for human agents without the need for human intervention, is of particular concern here since it is not difficult to imagine that such systems will inevitably encounter ethically salient contexts where an ethical decision must be made. As agentic AIs begin to be deployed across various domains, a trend which we can expect to continue along with progress in AI capabilities, it is crucial to consider the relevant emerging ethical questions. Accordingly, this post will raise some preliminary thoughts and questions around the regulation of agentic AI systems that are capable of engaging in ethical decision-making and reaching an ethical conclusion, or as they are often referred to in the machine ethics literature: artificial moral agents. It will also raise more questions than answers, but this is deliberate. We are in the early stages of the development of such agents, and we certainly don’t have all the answers at present, but we should be proactive in our thinking about the ethical questions rather than waiting for problems to arise!

Background

It will be useful to first provide some brief background and set-up, especially since many readers might not be familiar with topics in machine ethics. Machine ethics is the area of AI ethics research that is concerned with questions regarding the design of AI systems with ethical decision-making capabilities or artificial moral agents (AMAs). One set of questions in this area are taxonomic: AMAs could vary in the robustness of their ethical capacities, and ethical questions about the design and deployment of such systems will vary accordingly. For instance, one influential taxonomy divides AMAs into implicit, explicit, and full AMAs. For the sake of brevity, we need not get into the details of this taxonomic scheme, but read more in this article.

A second set of questions in machine ethics concerns design approaches to developing AMAs. Top-down approaches involve converting normative ethical theories about ethically appropriate behavior into algorithmic decision procedures for AIs to implement, and bottom-up approaches involve training an AMA to behave ethically through learning from cases, which could resemble the moral education of a child that learns right from wrong on the basis of lots of experience. Finally, hybrid design approaches involve a mix of the first two approaches.

A last set of questions concerns the potential risks and benefits of developing (or attempting to develop) AMAs. There is ongoing debate of course, but it need not concern us here since that isn’t the focus of this blog post. Interested readers can learn more about these debates.

Now, to complete the set-up, recent studies of the ethical reasoning capabilities of large language models (LLMs) reveal that many people regard ethical advice supplied by LLMs as superior in various ways to human advice, along the dimensions of trustworthiness, thoughtfulness, and correctness. In addition, AI systems are increasingly being deployed within a wide range of ethically salient domains of application, including healthcare, criminal justice, the military, and so on. Taken together, these observations raise a number of interesting philosophical questions about AI, of course, many of which are beginning to receive treatment in the AI ethics literature: what are the ethical capacities of current AI systems? Should we trust their ethical outputs? Is it possible to design sophisticated AMAs that can replicate human-level ethical reasoning or even improve upon our reasoning? And so on. However, the relationship between the machine ethics project of developing AMAs and AI regulation and policy-making is not an area that has been thoroughly investigated yet. This lacuna in the AI ethics and policy space should begin to be filled, lest we succeed at developing advanced AMAs (or believe we have succeeded: an important distinction!) before sufficient regulatory mechanisms have been crafted to address such matters.

Relevant Regulatory Questions (preliminary and non-exhaustive!)

Now, we can proceed to considering some possible regulatory questions around AMAs. As the title of this section makes clear, these are merely preliminary and certainly non-exhaustive, but the aim is to give readers some sense of the kinds of questions we will be facing as AMAs become increasingly advanced.

First, explicit AMAs are AI systems that are capable of ethical reasoning involving the usage of ethical concepts, such as fairness, beneficence, wrongness, etc. It is controversial whether or not current large language models (LLMs) possess this capacity, though they do include normative language in their outputs. That is, if they are prompted with an ethical question, they will generate responses that include ethical terms. Some machine ethicists have claimed that as AI systems become increasingly deployed in ethically salient domains, they will need to be designed with ethical reasoning capabilities in order to ensure ethically appropriate outputs. Consider, for instance, the kinds of decisions that could be made by autonomous vehicles and care bots. Some of these are certainly ethically relevant! In view of this, here are some relevant regulatory questions:

(i) How should explicit AMAs be regulated in connection with their various domains of application? Ethical questions and stakes will of course differ among domains of application, which should be taken into consideration, and it is far from obvious how we should program AMAs to behave across all possible scenarios. They need to be flexible or adaptive in their ethical decision-making, and we need to be thinking about how to appropriately regulate such matters. (ii) Should some domains be strictly forbidden for AMAs to enter into? This question is motivated by the fact that we might not be comfortable with having AMAs make all kinds of ethical decisions, and there might be some types of decisions that we think (or at least currently think, given the state of the technology) should not be made by AI systems but should be left to human agents. (iii) Which types of AMAs should AI developers be prohibited from creating, given how risky they might be? Full AMAs, which would resemble full-fledged human moral agents that are capable of considering their own goals and values, are currently hypothetical, but these deserve at least some mention here since things are progressing rapidly! Some might argue that developers should be prohibited from creating such systems, while others might argue that they should be permitted to create them as long as developers comply with some currently non-existing regulatory mechanisms.

A second category of regulatory questions arises in relation to design approaches in machine ethics. Top-down approaches, for example, which again involve ethical theory-centered design where AIs are designed to satisfy particular ethical theoretical criteria (e.g. the classical act-utilitarian criterion: maximize happiness!), raise interesting questions, and so do bottom-up approaches involving case-based learning for developing ethically competent AIs (e.g. through supervised and/or reinforcement learning). Here are some regulatory questions we would presumably need to address for these approaches:

(i) How should the design of AMAs be regulated? Should it be required that certain ethical values or principles be realized to a sufficient degree by AMAs, or should this be left up to the developers? Of course some regulatory constraints would be necessary here, but just how much is too much could be a difficult question in this context, as there is a danger of paternalistically imposing certain ethical values onto developers. (ii) Won't there be lots of normative disagreement about how AMAs should be designed, given the diversity of values in our pluralistic society, and how we should be regulating the design of these AIs? These are incredibly difficult questions that we will have to face!

Finally, there are many possible risks and benefits of developing AMAs, and we will need to determine to what extent regulation is necessary for ensuring that the deployment of AMAs is beneficial to society. Each kind of risk will perhaps require some corresponding regulation to protect us from the risk, but this is easier said than done when it comes to figuring out how this could be done effectively. How might AI regulation help to protect us from the risks while also promoting the benefits of AMAs? And how should we think about holding AI developers responsible if/when autonomous AMAs make mistakes, especially as their autonomous capacities increase and they become capable of performing more and more tasks without significant human oversight? These are the types of questions we will be facing as AI progress continues and specifically as AMAs become increasingly ethically sophisticated. Let’s start considering them now before the technology outpaces us!