Vision-based excavator pose estimation for automatic control

Calibration of visual measurement system for excavator manipulator pose

“…Rarely are these pose values obtained through deep learning used as feedback signals for operation trajectory tracking control systems. This difference in application purposes leads to variations in pose measurement methods, such as (1) the definition of 'pose', which, in the context of deep learning-based excavator pose estimation, typically represents the pose as pixel coordinates of the excavator's joints in image space [21], akin to human pose estimation, whereas for the automatic control of operation trajectories, pose is defined as the angles of the links in the operational space [25]; and (2) the perspective of pose estimation, where deep learning-based excavator pose estimation generally studies generic datasets and employs a surveillance camera perspective, whereas the automatic control of operation trajectories requires real-time operational imagery, captured from camera perspectives that can rotate and move along with the excavator.…”

“…Based on the above, due to limitations in fundamental conditions such as application purposes, measurement accuracy, the generality of training sets, as well as the quantity and quality of these datasets, the application of deep learning-based methods for measuring the pose of excavator manipulators in the automatic control of operational trajectories faces many restrictions [25]. Therefore, traditional marker-based methods, including AprilTag, Aruco markers, and other customized markers [26], continue to be widely utilized in this field [25]. These methods yield feature point pixel coordinates (FPPCs), and the process of generating the pose often involves mapping the relationship between FPPC and the corresponding manipulator pose, referred to as the 'pose mapping relationship'.…”