Teaching a Bipedal Robot to Walk

Authors:

Anja Vujanac, 4th-year student at the Gymnasium in Raška

Tatjana Imbrišić, 4th-year student

Mentors:

Nataša Jovanović, graduate engineer in electrical and computer engineering, PFE

Dimitrije Pešić, School of Electrical Engineering, University of Belgrade

Abstract

The Gymnasium environment BipedalWalker-v3 was used to compare three algorithms: Q-Learning, Twin Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC). Due to the continuous nature of the problem, Q-Learning proved to be inadequate. TD3 achieved the best results thanks to its ability to work with continuous actions. The implementation of the SAC algorithm in this study failed to achieve the expected performance. TD3 proved to be the most suitable algorithm for solving this problem, although instability in its behavior was observed on more challenging terrain.

Full paper