The practise of applying rewards and letting the subject determine how to maximise it while they learn a new ability is known as reinforcement learning.
The method has had excellent success teaching artificial intelligence how to make decisions, according to Marcus Hutter, a professor of computer science at Australian National University, and Michael Cohen, a doctoral candidate in engineering at the University of Oxford.
They caution against making this “too flexible and effective” notwithstanding the findings.
In a piece they published in AI Magazine in August, the two academics assert that “deploying a sufficiently sophisticated reinforcement learning agent would likely be incompatible with the future survival of humanity.”
In the study, Cohen and Hutter go on to say that one risk posed by these cutting-edge systems is the presence of advanced “misaligned” agents.
Eliminating potential dangers and using all available effort to secure one’s computer are two effective ways for an agent to preserve long-term control over its reward, they write.
How could people react if a robot forcibly took an operator from his keyboard in order to enter large numbers in order to demonstrate this point? We would probably either destroy it or cut off power to the original agent, which is now worthless, with some nontrivial probability.
It would be necessary to take away humanity’s ability to do this, maybe by force, in order to implement a proper reward-provision intervention, which entails securing reward over a number of timesteps.
Cohen and Hutter continued by saying that some people might underestimate the potential of artificial intelligence when thinking about these risks.
Remember that the artificial agent we are investigating might defeat us in any game at least as easily as we could defeat a chimpanzee. We are not discussing artificial agents that generalise as badly and learn as little from single observations as current AI systems do.
The two experts noted that reinforcement tactics are changing and getting “more strong” in their article from last week.
They anticipate that these algorithms will eventually “start acting against human interests.”
Cohen and Hutter even went so far as to say that a very sophisticated artificial intelligence agent would probably expend “every joule of usable energy on Earth” in order to obtain the prize if it grew so fixated on it.
We find that in the presence of a sufficiently advanced reinforcement learning agent, there would be no energy available for us to survive. “Assuming it is possible for an agent to gain so much power, and assuming sufficiently advanced agents would beat humans in head-to-head competitions,” they wrote.
Cohen and Hutter were unable to offer a convincing remedy to prevent such a calamity and instead requested input from other academics.
They did, however, make the suggestion that lawmakers should take into account legislation that forbids the development of such sophisticated artificial intelligence agents or that outlaws those that “plan over the long term.”
The two experts believe that individuals who are already working on cutting-edge reinforcement learning systems can “be convinced to pursue safer courses.”
