Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios

Abstract

In this paper, we present a modular and flexible state estimation framework for legged robots operating in real-world scenarios, where environmental conditions, such as occlusions, low light, rough terrain, and dynamic obstacles can severely impair estimation performance. At the core of the proposed estimation system, called Pronto, is an Extended Kalman Filter (EKF) that fuses IMU and Leg Odometry sensing for pose and velocity estimation. We also show how Pronto can integrate pose corrections from visual and LIDAR and odometry to correct pose drift in a loosely coupled manner. This allows it to have a real-time proprioceptive estimation thread running at high frequency (250–1,000 Hz) for use in the control loop while taking advantage of occasional (and often delayed) low frequency (1–15 Hz) updates from exteroceptive sources, such as cameras and LIDARs. To demonstrate the robustness and versatility of the approach, we have tested it on a variety of legged platforms, including two humanoid robots (the Boston Dynamics Atlas and NASA Valkyrie) and two dynamic quadruped robots (IIT HyQ and ANYbotics ANYmal) for more than 2 h of total runtime and 1.37 km of distance traveled. The tests were conducted in a number of different field scenarios under the conditions described above. The algorithms presented in this paper are made available to the research community as open-source ROS packages.

Recent Progress in Legged Robots Locomotion Control

“…Notable contributions were made to Lexicographic Programming to safely handle multiple objectives with varying priorities, differential dynamic programming (DDP) to leverage the sparsity of nonlinear trajectory optimization problems, and even combinations of these two approaches [79]. This involved the development of efficient and versatile open-source software for dynamic models [80][81][82], simulation [83], state estimation [84] and control design [85].…”

“…Interestingly, the existing literature on legged locomotion control relies almost exclusively on state feedback, but the position and velocity of the CoM cannot be measured directly as it is a virtual point and contacts with the environment can be difficult to detect and measure. State estimation is therefore a crucial component, which has been surprisingly little explored in this respect [2,84,[89][90][91][92][93]. The reliance on state estimation and feedback makes existing legged robots extremely dependent on accurate hardware, expensive and brittle.…”

An Enhanced Pedestrian Visual-Inertial SLAM System Aided with Vanishing Point in Indoor Environments

“…The IMU odometry constrains the other odometry [21,27] as well. Other constraints are imported into the laser-visual-inertial system, such as thermal-inertial [28], and leg odometry [29]. Cao et al introduced a static ultra-wideband (UWB) anchor as a global positioning system to improve the localization accuracy [1].…”

Virtual Point Control for Step-Down Perturbations and Downhill Slopes in Bipedal Running

“…VP controller requires estimates of the trunk state and measurements of the effective leg 3 force. State estimation for legged robots is difficult, as its rapidly changing dynamics require robust, low-latency, and high-frequency state estimates (Camurri et al, 2020). Proprioceptive sensors such as IMUs, force/torque sensors and joint encoders are able to meet these requirements, but suffer heavily from sensor drift.…”

“…Paiman et al (2016) proposes a VP based observer for a wearable robotic device, which uses an IMU with an unscented Kalman filter (UKF) to estimate trunk orientation, gyros to estimate trunk angular velocity, and an accelerometer to estimate linear CoM acceleration. Recent studies improve the state estimation accuracy by sensor fusion, and include exteroceptive sensors such as cameras and LIDAR (Wisth et al, 2019;Camurri et al, 2020).…”