Ghent University Academic Bibliography

Advanced

Incorporating prior knowledge into deep neural network controllers of legged robots

Jonas Degrave UGent (2018)
abstract
This dissertation explores multiple ways of adding prior knowledge to neural networks used as controllers in robotics. It can largely be split into two parts. The first part of the dissertation focuses on adding prior knowledge to the gait generation, to spend less time in the optimization process to find efficient solutions. The second part of the dissertation focuses on the use of morphological computation as prior knowledge in the generation of stable gaits for legged robots. In the introduction, we discuss in depth what is meant by prior knowledge in this context. We show how the concept of prior evidence emerges naturally from the creation of a probabilistic framework for ‘degree of belief’. We then discuss how this prior evidence can be used in robotics using a paradigm built on an alternative view on computation, called morphological computation. We argue how this approach makes a natural match for controlling compliant robots. There are multiple ways to add prior knowledge to neural networks. As a first step, in chapter 2 we explored data augmentation as a way to teach neural networks how to be invariant to affine image transformations. Image augmentations are a known way to have a convolutional neural network learn this invariance in natural images. We improve on this idea by putting the affine transform as a differentiable layer into 8 the neural network, thereby allowing the neural network to encode this invariance explicitly, rather than to have to encode this implicitly in the values of its parameters. The network is then able to transform images as a special type of layer, next to convolutional or dense layers. We show that explicitly encoding this prior knowledge of affine invariance into the architecture outperforms the previous method of using image augmentations. Next, in chapter 3 we move our focus to robotics and develop three different gaits for the quadrupedal compliant robot Oncilla: a sinebased approach, a biologically inspired half ellipse approach and a spline-based approach. After comparing these approaches, we find that the method based on biological gaits is the most efficient of the three, especially at higher speeds. After this, we move our attention to approaches for turning. We showe the importance of scapulae for turning in quadrupedal robots. We also show that to be able to optimize the gaits without relying on a model, a lot of prior knowledge is needed to keep the time required for gait optimization low. Consequently, in chapter 4 we evaluate whether transfer learning known gaits to gaits for new situations improves the optimization process. We analyzed this by starting the optimization process for various setups with gait parameters which had already been optimized for flat terrain. We find that it indeed works in most cases, and at least did not hurt the optimization process. We uncover that in this case, the reduced amount of exploration of the parameter space required before the parameters converges to an optimal solution is the reason for a warm start helping the optimization process. The optimization algorithm can, therefore, find good solutions faster, and fine-tune the parameters longer for a better end performance. After this, we move our focus to morphological computation. As a first aspect in chapter 5, we study morphological sensing, and more specifically, whether we can use general purpose sensors available on a small legged robot to classify the underground it is walking on. Since the dynamics of the robot change with the underground it walks across, it should be possible to infer this underground from the sensors monitoring the body of the robot. Since we do not require any specialized sensors for the detection of the underground, we can argue that we are using the body of the robot as a resource of computation for the classification. We can indeed classify the underground successfully in 9 most cases, both with supervised and unsupervised algorithms. In a second part of chapter 5, we delve into which properties of the models are important for the correct classification. We find indications in our data that both memory and non-linearities are important aspects of this classification process and that they reinforce each other, which provides a starting point for the research in the next chapter. Since gaits of legged robots are typically on the eigenfrequencies of their morphology, the morphology can probably be used as a resource for computation to generate the control signals. This is a concept called morphological control. In chapter 6 we are indeed able to move part of the control onto the morphology and show that there is a trade-off between the memory aspects and the non-linear dynamics needed for it to perform well. It seems that the main parameter is the number of uncorrelated signals the linear regression receives. The more signals with information, the better the performance and the smaller the error between the found closed loop controller and the target open loop trajectory. Using this, we are able to have the Oncilla perform a stable gait without requiring any memory, using an ELM setup to generate the motor signals from the sensors. We show that a stable closed loop limit cycle can be obtained using supervised learning for only a few periods of its gait, slowly transferring control from the open to the closed loop. Finally, in chapter 7 we stretch the idea of morphological computation, and treat the whole legged robot with its controller as a single system to be optimized. We optimize it using a deep neural network as controller of a system by backpropagation through physics. To do this, we developed a physics engine framework inside an automatic differentiation library. This allows us to backpropagate through the controller, physics and renderer. We are able to show remarkably short optimization processes despite only having quite complex sensory signals such as cameras as inputs, in setups which are only partially observable and underactuated. We conclude that incorporating prior knowledge is beneficial when setting up machine learning models for controlling robots. We also conclude that we were able to show that both morphological sensing and morphological control can be valid strategies for developing controllers for legged robots.
Please use this url to cite or link to this publication:
author
promoter
UGent
organization
alternative title
Incorporatie van a-priorikennis in diepe neurale netwerken voor regelaars van robots met poten
year
type
dissertation
publication status
published
subject
pages
186 pages
defense location
Ghent
defense date
2018-05-15 18:00
ISBN
978-94-6355-111-3
language
English
UGent publication?
yes
classification
D1
copyright statement
Creative Commons Public Domain Dedication (CC0 1.0)
id
8562053
handle
http://hdl.handle.net/1854/LU-8562053
date created
2018-05-16 20:35:13
date last changed
2018-06-19 11:54:06
@phdthesis{8562053,
  abstract     = {This dissertation explores multiple ways of adding prior knowledge
to neural networks used as controllers in robotics. It can largely be
split into two parts. The first part of the dissertation focuses on
adding prior knowledge to the gait generation, to spend less time in
the optimization process to find efficient solutions. The second part
of the dissertation focuses on the use of morphological computation
as prior knowledge in the generation of stable gaits for legged robots.
In the introduction, we discuss in depth what is meant by prior knowledge
in this context. We show how the concept of prior evidence
emerges naturally from the creation of a probabilistic framework for
{\textquoteleft}degree of belief{\textquoteright}. We then discuss how this prior evidence can be used
in robotics using a paradigm built on an alternative view on computation,
called morphological computation. We argue how this approach
makes a natural match for controlling compliant robots.
There are multiple ways to add prior knowledge to neural networks.
As a first step, in chapter 2 we explored data augmentation as a way to
teach neural networks how to be invariant to affine image transformations.
Image augmentations are a known way to have a convolutional
neural network learn this invariance in natural images. We improve on
this idea by putting the affine transform as a differentiable layer into
8
the neural network, thereby allowing the neural network to encode
this invariance explicitly, rather than to have to encode this implicitly
in the values of its parameters. The network is then able to transform
images as a special type of layer, next to convolutional or dense layers.
We show that explicitly encoding this prior knowledge of affine
invariance into the architecture outperforms the previous method of
using image augmentations.
Next, in chapter 3 we move our focus to robotics and develop three
different gaits for the quadrupedal compliant robot Oncilla: a sinebased
approach, a biologically inspired half ellipse approach and a
spline-based approach. After comparing these approaches, we find
that the method based on biological gaits is the most efficient of the
three, especially at higher speeds. After this, we move our attention
to approaches for turning. We showe the importance of scapulae for
turning in quadrupedal robots. We also show that to be able to optimize
the gaits without relying on a model, a lot of prior knowledge is
needed to keep the time required for gait optimization low.
Consequently, in chapter 4 we evaluate whether transfer learning
known gaits to gaits for new situations improves the optimization process.
We analyzed this by starting the optimization process for various
setups with gait parameters which had already been optimized for flat
terrain. We find that it indeed works in most cases, and at least did
not hurt the optimization process. We uncover that in this case, the
reduced amount of exploration of the parameter space required before
the parameters converges to an optimal solution is the reason for a
warm start helping the optimization process. The optimization algorithm
can, therefore, find good solutions faster, and fine-tune the
parameters longer for a better end performance.
After this, we move our focus to morphological computation. As a first
aspect in chapter 5, we study morphological sensing, and more specifically,
whether we can use general purpose sensors available on a small
legged robot to classify the underground it is walking on. Since the
dynamics of the robot change with the underground it walks across,
it should be possible to infer this underground from the sensors monitoring
the body of the robot. Since we do not require any specialized
sensors for the detection of the underground, we can argue that we
are using the body of the robot as a resource of computation for the
classification. We can indeed classify the underground successfully in
9
most cases, both with supervised and unsupervised algorithms. In a
second part of chapter 5, we delve into which properties of the models
are important for the correct classification. We find indications in our
data that both memory and non-linearities are important aspects of
this classification process and that they reinforce each other, which
provides a starting point for the research in the next chapter.
Since gaits of legged robots are typically on the eigenfrequencies of
their morphology, the morphology can probably be used as a resource
for computation to generate the control signals. This is a concept
called morphological control. In chapter 6 we are indeed able to move
part of the control onto the morphology and show that there is a
trade-off between the memory aspects and the non-linear dynamics
needed for it to perform well. It seems that the main parameter
is the number of uncorrelated signals the linear regression receives.
The more signals with information, the better the performance and
the smaller the error between the found closed loop controller and
the target open loop trajectory. Using this, we are able to have the
Oncilla perform a stable gait without requiring any memory, using an
ELM setup to generate the motor signals from the sensors. We show
that a stable closed loop limit cycle can be obtained using supervised
learning for only a few periods of its gait, slowly transferring control
from the open to the closed loop.
Finally, in chapter 7 we stretch the idea of morphological computation,
and treat the whole legged robot with its controller as a single system
to be optimized. We optimize it using a deep neural network as controller
of a system by backpropagation through physics. To do this, we
developed a physics engine framework inside an automatic differentiation
library. This allows us to backpropagate through the controller,
physics and renderer. We are able to show remarkably short optimization
processes despite only having quite complex sensory signals such
as cameras as inputs, in setups which are only partially observable
and underactuated.
We conclude that incorporating prior knowledge is beneficial when
setting up machine learning models for controlling robots. We also
conclude that we were able to show that both morphological sensing
and morphological control can be valid strategies for developing
controllers for legged robots.},
  author       = {Degrave, Jonas},
  isbn         = {978-94-6355-111-3},
  language     = {eng},
  pages        = {186},
  school       = {Ghent University},
  title        = {Incorporating prior knowledge into deep neural network controllers of legged robots},
  year         = {2018},
}

Chicago
Degrave, Jonas. 2018. “Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots.”
APA
Degrave, J. (2018). Incorporating prior knowledge into deep neural network controllers of legged robots.
Vancouver
1.
Degrave J. Incorporating prior knowledge into deep neural network controllers of legged robots. 2018.
MLA
Degrave, Jonas. “Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots.” 2018 : n. pag. Print.