The optimisation method known as evolution strategy (ES) has been used to solve a variety of complex decision-making issues, including the control of power systems and the movement of legs.
The challenge of scaling to problems requiring high-dimensional sensory inputs for encoding dynamics, such as training robots with complex visual inputs, is one of the fundamental drawbacks of ES-based algorithms, nevertheless.
To address this issue, Google AI researchers have created a learning system that effectively solves high dimensional issues by combining ES and representation learning.
The main concept, according to the article titled “PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations,” is to use predictive information to obtain a representation of the high-dimensional environment dynamics.
The learnt compact representation would be translated into robot actions using the same innovation on Augmented Random Search (ARS), a well-known ES algorithm.
The scientists first put PI-ARS to the test on the tricky problem of visual-locomotion for legged robots. The programme can navigate a range of challenging settings and enables quick training of effective vision-based locomotion controllers.
After that, the researchers merged PI (Predictive Information) with Augmented Random Search (ARS), an algorithm with outstanding optimization skills for difficult decision-making tasks. “At each iteration of ARS, it samples a population of perturbed controller parameters, evaluates their performance in the testing environment, and then computes a gradient that shifts the controller towards the ones that performed better,” the blog stated.
