Emergence of Prediction in Delayed Reaching Task Through Neuroevolution

Abstract

Prediction is an important foundation of cognitive and intelligent behavior. Recent advances in deep learning heavily depend on prediction, in the form of self-supervised learning based on prediction and reinforcement learning (reward prediction). However, how such predictive capabilities emerged from simple organisms has not been investigated fully. Prior works have shown the relationship between input delay and predictive function to compensate for such delay.

In this thesis, we investigate the emergence of predictive capabilities in simple evolving neural network controllers, where not only the connection weights but also the network topology evolves. We focus on two main research questions: (1) what fitness criterion promotes predictive behavior? and (2) what changes in the neural network structure correlate with predictive function? To test this, we set up a delayed reaching task, where a two-segment arm is controlled to reach a moving target where the target’s location arrives at the arm’s controller with a period of delay. The arm also has an option of picking up a stick (a tool) to extend its reach. We tested several factors to be included in the fitness function: (1) energy usage, (2) tracking target, and (3) number of tool pick-ups. Our results show that minimizing energy usage is a key to the emergence of 1 prediction. As for the evolved network structure, we found that controllers with more recurrent loops perform better in the task, i.e., tracking the predicted location of the moving target. These results lend us two important insights regarding the evolutionary emergence of prediction: (1) energy minimization is a key driving force, without which random strategy (wasteful in terms of energy) can potentially perform equally, and (2) recurrent loops in the neural network controller not only play the traditional role of memory, but they also serve the purpose of prediction. We expect our results to shed new light on the origin of neural architectures supporting prediction and the energetic constraints.