### Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots

(2018)- abstract
- This dissertation explores multiple ways of adding prior knowledge to neural networks used as controllers in robotics. It can largely be split into two parts. The first part of the dissertation focuses on adding prior knowledge to the gait generation, to spend less time in the optimization process to find efficient solutions. The second part of the dissertation focuses on the use of morphological computation as prior knowledge in the generation of stable gaits for legged robots. In the introduction, we discuss in depth what is meant by prior knowledge in this context. We show how the concept of prior evidence emerges naturally from the creation of a probabilistic framework for ‘degree of belief’. We then discuss how this prior evidence can be used in robotics using a paradigm built on an alternative view on computation, called morphological computation. We argue how this approach makes a natural match for controlling compliant robots. There are multiple ways to add prior knowledge to neural networks. As a first step, in chapter 2 we explored data augmentation as a way to teach neural networks how to be invariant to affine image transformations. Image augmentations are a known way to have a convolutional neural network learn this invariance in natural images. We improve on this idea by putting the affine transform as a differentiable layer into 8 the neural network, thereby allowing the neural network to encode this invariance explicitly, rather than to have to encode this implicitly in the values of its parameters. The network is then able to transform images as a special type of layer, next to convolutional or dense layers. We show that explicitly encoding this prior knowledge of affine invariance into the architecture outperforms the previous method of using image augmentations. Next, in chapter 3 we move our focus to robotics and develop three different gaits for the quadrupedal compliant robot Oncilla: a sinebased approach, a biologically inspired half ellipse approach and a spline-based approach. After comparing these approaches, we find that the method based on biological gaits is the most efficient of the three, especially at higher speeds. After this, we move our attention to approaches for turning. We showe the importance of scapulae for turning in quadrupedal robots. We also show that to be able to optimize the gaits without relying on a model, a lot of prior knowledge is needed to keep the time required for gait optimization low. Consequently, in chapter 4 we evaluate whether transfer learning known gaits to gaits for new situations improves the optimization process. We analyzed this by starting the optimization process for various setups with gait parameters which had already been optimized for flat terrain. We find that it indeed works in most cases, and at least did not hurt the optimization process. We uncover that in this case, the reduced amount of exploration of the parameter space required before the parameters converges to an optimal solution is the reason for a warm start helping the optimization process. The optimization algorithm can, therefore, find good solutions faster, and fine-tune the parameters longer for a better end performance. After this, we move our focus to morphological computation. As a first aspect in chapter 5, we study morphological sensing, and more specifically, whether we can use general purpose sensors available on a small legged robot to classify the underground it is walking on. Since the dynamics of the robot change with the underground it walks across, it should be possible to infer this underground from the sensors monitoring the body of the robot. Since we do not require any specialized sensors for the detection of the underground, we can argue that we are using the body of the robot as a resource of computation for the classification. We can indeed classify the underground successfully in 9 most cases, both with supervised and unsupervised algorithms. In a second part of chapter 5, we delve into which properties of the models are important for the correct classification. We find indications in our data that both memory and non-linearities are important aspects of this classification process and that they reinforce each other, which provides a starting point for the research in the next chapter. Since gaits of legged robots are typically on the eigenfrequencies of their morphology, the morphology can probably be used as a resource for computation to generate the control signals. This is a concept called morphological control. In chapter 6 we are indeed able to move part of the control onto the morphology and show that there is a trade-off between the memory aspects and the non-linear dynamics needed for it to perform well. It seems that the main parameter is the number of uncorrelated signals the linear regression receives. The more signals with information, the better the performance and the smaller the error between the found closed loop controller and the target open loop trajectory. Using this, we are able to have the Oncilla perform a stable gait without requiring any memory, using an ELM setup to generate the motor signals from the sensors. We show that a stable closed loop limit cycle can be obtained using supervised learning for only a few periods of its gait, slowly transferring control from the open to the closed loop. Finally, in chapter 7 we stretch the idea of morphological computation, and treat the whole legged robot with its controller as a single system to be optimized. We optimize it using a deep neural network as controller of a system by backpropagation through physics. To do this, we developed a physics engine framework inside an automatic differentiation library. This allows us to backpropagate through the controller, physics and renderer. We are able to show remarkably short optimization processes despite only having quite complex sensory signals such as cameras as inputs, in setups which are only partially observable and underactuated. We conclude that incorporating prior knowledge is beneficial when setting up machine learning models for controlling robots. We also conclude that we were able to show that both morphological sensing and morphological control can be valid strategies for developing controllers for legged robots.

Please use this url to cite or link to this publication:
http://hdl.handle.net/1854/LU-8562053

- author
- Jonas Degrave UGent
- promoter
- Francis wyffels UGent
- organization
- alternative title
- Incorporatie van a-priorikennis in diepe neurale netwerken voor regelaars van robots met poten
- year
- 2018
- type
- dissertation
- publication status
- published
- subject
- pages
- 186 pages
- defense location
- Ghent
- defense date
- 2018-05-15 18:00
- UGent publication?
- yes
- classification
- U
- copyright statement
*Creative Commons Public Domain Dedication (CC0 1.0)*- id
- 8562053
- handle
- http://hdl.handle.net/1854/LU-8562053
- date created
- 2018-05-16 20:35:13
- date last changed
- 2018-05-16 20:37:12

@phdthesis{8562053, abstract = {This dissertation explores multiple ways of adding prior knowledge to neural networks used as controllers in robotics. It can largely be split into two parts. The first part of the dissertation focuses on adding prior knowledge to the gait generation, to spend less time in the optimization process to find efficient solutions. The second part of the dissertation focuses on the use of morphological computation as prior knowledge in the generation of stable gaits for legged robots. In the introduction, we discuss in depth what is meant by prior knowledge in this context. We show how the concept of prior evidence emerges naturally from the creation of a probabilistic framework for {\textquoteleft}degree of belief{\textquoteright}. We then discuss how this prior evidence can be used in robotics using a paradigm built on an alternative view on computation, called morphological computation. We argue how this approach makes a natural match for controlling compliant robots. There are multiple ways to add prior knowledge to neural networks. As a first step, in chapter 2 we explored data augmentation as a way to teach neural networks how to be invariant to affine image transformations. Image augmentations are a known way to have a convolutional neural network learn this invariance in natural images. We improve on this idea by putting the affine transform as a differentiable layer into 8 the neural network, thereby allowing the neural network to encode this invariance explicitly, rather than to have to encode this implicitly in the values of its parameters. The network is then able to transform images as a special type of layer, next to convolutional or dense layers. We show that explicitly encoding this prior knowledge of affine invariance into the architecture outperforms the previous method of using image augmentations. Next, in chapter 3 we move our focus to robotics and develop three different gaits for the quadrupedal compliant robot Oncilla: a sinebased approach, a biologically inspired half ellipse approach and a spline-based approach. After comparing these approaches, we find that the method based on biological gaits is the most efficient of the three, especially at higher speeds. After this, we move our attention to approaches for turning. We showe the importance of scapulae for turning in quadrupedal robots. We also show that to be able to optimize the gaits without relying on a model, a lot of prior knowledge is needed to keep the time required for gait optimization low. Consequently, in chapter 4 we evaluate whether transfer learning known gaits to gaits for new situations improves the optimization process. We analyzed this by starting the optimization process for various setups with gait parameters which had already been optimized for flat terrain. We find that it indeed works in most cases, and at least did not hurt the optimization process. We uncover that in this case, the reduced amount of exploration of the parameter space required before the parameters converges to an optimal solution is the reason for a warm start helping the optimization process. The optimization algorithm can, therefore, find good solutions faster, and fine-tune the parameters longer for a better end performance. After this, we move our focus to morphological computation. As a first aspect in chapter 5, we study morphological sensing, and more specifically, whether we can use general purpose sensors available on a small legged robot to classify the underground it is walking on. Since the dynamics of the robot change with the underground it walks across, it should be possible to infer this underground from the sensors monitoring the body of the robot. Since we do not require any specialized sensors for the detection of the underground, we can argue that we are using the body of the robot as a resource of computation for the classification. We can indeed classify the underground successfully in 9 most cases, both with supervised and unsupervised algorithms. In a second part of chapter 5, we delve into which properties of the models are important for the correct classification. We find indications in our data that both memory and non-linearities are important aspects of this classification process and that they reinforce each other, which provides a starting point for the research in the next chapter. Since gaits of legged robots are typically on the eigenfrequencies of their morphology, the morphology can probably be used as a resource for computation to generate the control signals. This is a concept called morphological control. In chapter 6 we are indeed able to move part of the control onto the morphology and show that there is a trade-off between the memory aspects and the non-linear dynamics needed for it to perform well. It seems that the main parameter is the number of uncorrelated signals the linear regression receives. The more signals with information, the better the performance and the smaller the error between the found closed loop controller and the target open loop trajectory. Using this, we are able to have the Oncilla perform a stable gait without requiring any memory, using an ELM setup to generate the motor signals from the sensors. We show that a stable closed loop limit cycle can be obtained using supervised learning for only a few periods of its gait, slowly transferring control from the open to the closed loop. Finally, in chapter 7 we stretch the idea of morphological computation, and treat the whole legged robot with its controller as a single system to be optimized. We optimize it using a deep neural network as controller of a system by backpropagation through physics. To do this, we developed a physics engine framework inside an automatic differentiation library. This allows us to backpropagate through the controller, physics and renderer. We are able to show remarkably short optimization processes despite only having quite complex sensory signals such as cameras as inputs, in setups which are only partially observable and underactuated. We conclude that incorporating prior knowledge is beneficial when setting up machine learning models for controlling robots. We also conclude that we were able to show that both morphological sensing and morphological control can be valid strategies for developing controllers for legged robots.}, author = {Degrave, Jonas}, pages = {186}, school = {Ghent University}, title = {Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots}, year = {2018}, }

- Chicago
- Degrave, Jonas. 2018. “Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots.”
- APA
- Degrave, J. (2018).
*Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots*. - Vancouver
- 1.Degrave J. Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots. 2018.
- MLA
- Degrave, Jonas. “Incorporating Prior Knowledge into Deep Neural Network Controllers of Legged Robots.” 2018 : n. pag. Print.