Self-Organization In Brain-Inspired Robot Control

we want that robots can learn from interaction from essentially playing around the world in the same way as children are doing it.

I am working on learning algorithms for robots, not in the way where we say exactly what they need to do but they really should learn by themselves. and then in practice, we start simple things: how can a robot start to behave, how can you understand from behavior, what are the mechanisms that are behind the world

We want the robot to autonomously and independently explore the environment without specifying or telling them how to do it we just provide an internal mechanism without a big number of instructions but just tell them how their parts are correlated. And let the robot to understand the physics over the world in a way from experience. In this way, the robot is able to self-organize these sensory motor development. This also helps the robots to exploit its environment and find out how it can manipulate its environment including itself.

Self-organizing features starts from homeostasis, the ability to keep system stable, to homeokinesis, a deciding feature to drive innovation and action.

the brain body system of the robot is simulated by a closed-loop control with forward model and controller model teaching each other. Homeokinetic learning algorithms encourages these changes in its parameter dynamics and state dynamics. It drives the brain of the robot to find out new modes of behavior and ways to excite them. The basis of homeokinesis is minimizing the time-loop error which leads to a progressive destabilization of the system.

In homeokinesis, the prediction error and the time loop error are the two main objective functions to minimize for the model. And they’re related to one another by the Jacobian Matrix, which in robotic control has the effect of stabilizing or destabilizing the state change. And the eigenvectors of the Jacobian matrix extracts the dimension with main feature movement of the robot.

In self-organizing robot, homeokinesis functions as the brain for the robot which is considered as a higher level in robot control architecture. With the benefits of deep neural network, neural architecture of robot control proves that using diamond-like deep recurrent neural networks enable more complex exploratory behavior. The proposed architecture is composed of many layers each of which is a recurrent neural network. In every inner layer, the higher level sees the outputs from the previous layer as virtual actions and virtual states and then predict new state as well as re-estimate the previous state as inputs for the next layer.

The results of the DIAMOND method show that on the one hand deep homeokinesis generates complex solution for difficult task and environment; on the other hand, we see trade-offs between deeper controller model and exploratory behaviours, thus we are using more evaluation metrices to illustrate the richer features, such as: exploration rate, time-loop error, noise level, success rate

Because of the considerable uncertainty in the higher levels, we are still in the way to understand the connections between deeper layers in the DIAMOND model and how well the deep homeokinesis can explain the flow of information among neural assemblies in brain.

We explore this neural assemblies by giving guiding inputs at higher levels in a top-down manner to steer the system to desired behavior. The guiding input can be goal-based function or simply additional errors. This helps us to understanding the interactions between the neighbouring layers in terms of consequences and actions.

By combining homeokinesis with other learning algorithms, such as cross-motor teaching, cross-entropy learning and modern reinforcement learning, we discover the potential of homeokinesis in high-dimensionality reduction and sparse space search. From empowerment to intrinsically motivated learning to deep homeokinesis, the robots learn to generate more and more complex behaviours simply by playing around in the world and use its internal dynamics to sense, act and even feel.

I hope one day we will have more of these robots that can help in real life tasks. This is something that really have to learn from interaction and that cannot be pre-programmed.