From IEEE Spectrum:
Researchers Teaching Robots How to Best Reject Orders from Humans
The Three Laws of Robotics, from the 56th edition of the “Handbook of Robotics” (published in 2058), are as follows:* Dave: Open the pod bay doors, HAL.
Pretty straightforward, right? And it’s nice that obeying humans is in there at number two. Problem is, humans often act like idiots, and sometimes, obeying the second law without question is really not the best thing for a robot to do. Gordon Briggs and Matthias Scheutz, from Tufts University’s Human-Robot Interaction Lab, are trying to figure out how to develop mechanisms for robots to reject orders that it receives from humans, as long as the robots have a good enough excuse for doing so.
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.
In linguistic theory, there’s this idea that if someone asks you to do something, whether or not you really understand what they want in a context larger than the words themselves, depends on what are called “felicity conditions.” Felicity conditions reflect your understanding and capability of actually doing that thing, as opposed to just knowing what the words mean. For robots, the felicity conditions necessary for carrying out a task might look like this:
The first three felicity conditions are easy enough to understand, but let’s take a quick look at four and five. “Social role and obligation” is simply referring to whether the robot believes that the person telling it to do a thing has the authority to do so. “Normative permissibility” is a complicated way of saying that the robot shouldn’t do things that it knows are dangerous, or more accurately, that a thing is okay to do if the robot doesn’t know that it’s dangerous....MORE
- Knowledge: Do I know how to do X?
- Capacity: Am I physically able to do X now? Am I normally physically able to do X?
- Goal priority and timing: Am I able to do X right now?
- Social role and obligation: Am I obligated based on my social role to do X?
- Normative permissibility: Does it violate any normative principle to do X?
HAL: I'm sorry, Dave. I'm afraid I can't do that.