Isaac Asimov’s Law of Robotics:

A robot must not injure a human being, must obey orders from a human being, and only protect its own existence if it is consistent with the above.

But we do not know how to programme that at the moment.

We already build advanced agents like industrial robots, which will get smarter and smarter, one day exceeding us in intelligence, becoming super-intelligent.

(Super-intelligence, as I said in a previous post, means better than humans in nearly every knowledge domain).

So, here’s a key question:  how do we motivate intelligent agents to pursue values that are good for humans when they overtake us in knowledge? The problem is called “value loading”.

We already have machines that will soon be able to improve their own operating systems, i.e. rewrite their own code, using techniques that are simply too complex for us to track.

Thus an artificial intelligence, especially one that may be rapidly growing towards super-intelligence, is not just brainier than we are, but there is no reason for it to share that inherent caution we grandly call “wisdom”, which draws in equal parts on our human intelligence and our instinct for personal survival.

We are only safe if the super-intelligent agent is loaded with the right values.  Loading such complex values will be difficult and require extreme caution.

How to define those values correctly? How to avoid them being misinterpreted, intentionally or unintentionally as an AI advances into super-intelligence, well out of human cognitive reach?

Here’s one idea: monitored learning. You could call a Strict Parent approach.

Any advance in intelligence or new code might be monitored for corruption by unimproved intelligent agents from the previous generation which themselves have been validated by unimproved agents from the generation before. There would therefore be layers of supervision, using all sorts of tests to ensure that the next generation of advanced intelligence was not putting the loaded value at risk. If so, it would be suppressed or marginalised.

There would be a permanent potential constraint, holding back every advance, just like a human boss with the power to say: “No, I’m not sure. I’m not ready. We won’t do it.”

But it would be bad news if there was any kind of arms race to achieve super-intelligence. Every player in the race would have an incentive to speed up the process. 

That’s a real  issue for a planet full of rival nations and beliefs, some lethal, some archaic. They would have no time for Asimov’s law.

Feels like a serious problem. 

Besides, when you actually try to code for Asimov’s Law, you pretty soon find how fuzzy it is.

Jamie told me that some of the wealthiest people in the tech industries of the West coast were putting money into a fund to reward the embedding of Asimov’s Law into intelligent devices.

But I got the impression he was very sceptical about the idea.