CoLo: A Performance Evaluation System for Multi-robot Cooperative Localization Algorithms

Abstract

This thesis presents CoLo-a performance evaluation system for two-dimensional cooperative localization algorithms. Multi-robot system have been used in a wide range of applications and cooperative localization is one of the fundamental tasks for mobile multirobot systems. However, developing cooperative localization algorithm is complex and timeconsuming. CoLo is created to reduce cooperative localization algorithm development cycle time. The system consists of two main parts: a physical experiment (CoLo-PE) for data collection and a software analysis tool (CoLo-AT) using real-world datasets to evaluate the performances of users' cooperative localization algorithms. CoLo uses an intuitive algorithm framework to allow researchers to conveniently add their cooperative localization algorithms to it. Instead of creating simulations or designing a new robotic testbed from the ground up, researchers only needed to load their algorithms in CoLo-AT and analyze them using data collected from CoLo-PE. Also, CoLo is aimed to create a standard so that effective comparisons can be made across research on localization algorithm. This paper details the design and operation of the physical experiment (CoLo-PE) which provides users guidelines to create their own robotic testbed with a ROS-based, scalable and affordable robotic team. And the paper explains how the software analysis tool (CoLo-AT) tests algorithms by running simulation processes to recreate the experiment trials using compatible real-world datasets of odometry data, measurement data, and the related groundtruth ii data. Researchers can test their algorithms and compare them with other state-of-the-art algorithms in various settings. CoLo-AT provides insightful metrics, and graphs include location error for localization accuracy and trace of state covariance to detect over-confident estimation results. It also has an animated plot to show the estimated trajectory and actual trajectory of each robot, which presents an intuitive visualization of the algorithm. CoLo has been used in the development of a published localization algorithm, where it was used to test the performance of the algorithm and compared to other existing algorithms objectively. CoLo provided the performance results in various setting and saved much time in experimental validation, which enabled a more rapid algorithm design process.