Semantically Synchronizing Multiple-Camera Systems with Human Pose Estimation

Zhe, Zhang; Wang, Qianqian; Qin, Wenhu

doi:10.3390/s21072464

Cited by 6 publications

(3 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is substantial work [1,6,8] in the area of computer vision where pictures taken at different times are combined to form a composite picture. In this work, it is already known that the pictures were taken at different times.…”

Section: Related Workmentioning

confidence: 99%

Effect of Timing Error: A Case Study of Navigation Camera

Kulkarni¹,

Joshi²

2022

Preprint

View full text Add to dashboard Cite

We focus on the problem of timing errors in navigation camera as a case study in a broader problem of the effect of a timing error in cyber-physical systems. These systems rely on the requirement that certain things happen at the same time or certain things happen periodically at some period T . However, as these systems get more complex, timing errors can occur between the components thereby violating the assumption about events being simultaneous (or periodic). We consider the problem of a surgical navigation system where optical markers detected in the 2D pictures taken by two cameras are used to localize the markers in 3D space. A predefined array of such markers, known as a reference element, is used to navigate the corresponding CAD model of a surgical instrument on patient's images. The cameras rely on the assumption that the pictures from both cameras are taken exactly at the same time. If a timing error occurs then the instrument may have moved between the pictures. We find that, depending upon the location of the instrument, this can lead to a substantial error in the localization of the instrument. Specifically, we find that if the actual movement is δ then the observed movement may be as high as 5δ in the operating range of the camera. Furthermore, we also identify potential issues that could affect the error in case there are changes to the camera system or to the operating range.

show abstract

Section: Related Workmentioning

confidence: 99%

Effect of Timing Error: A Case Study of Navigation Camera

Kulkarni¹,

Joshi²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Sinha and Pollefeys [17] presented a method for analyzing the motion of silhouettes in multiple video streams. Takahashi et al [18] detect 2D human poses in videos for temporal synchronization, and in a recent study by Zhang et al [19], synchronization is acheived by minimizing an energy function related to epipolar distances. These studies are based on 2D human pose data, which might be less accurate and require further processing in order to calculate the 3D pose estimation for pairs of cameras.…”

Section: Related Workmentioning

confidence: 99%

Spatio-Temporal Calibration of Multiple Kinect Cameras Using 3D Human Pose

Eichler

Hel‐Or

Shimshoni

2022

Sensors

View full text Add to dashboard Cite

RGB and depth cameras are extensively used for the 3D tracking of human pose and motion. Typically, these cameras calculate a set of 3D points representing the human body as a skeletal structure. The tracking capabilities of a single camera are often affected by noise and inaccuracies due to occluded body parts. Multiple-camera setups offer a solution to maximize coverage of the captured human body and to minimize occlusions. According to best practices, fusing information across multiple cameras typically requires spatio-temporal calibration. First, the cameras must synchronize their internal clocks. This is typically performed by physically connecting the cameras to each other using an external device or cable. Second, the pose of each camera relative to the other cameras must be calculated (Extrinsic Calibration). The state-of-the-art methods use specialized calibration session and devices such as a checkerboard to perform calibration. In this paper, we introduce an approach to the spatio-temporal calibration of multiple cameras which is designed to run on-the-fly without specialized devices or equipment requiring only the motion of the human body in the scene. As an example, the system is implemented and evaluated using Microsoft Azure Kinect. The study shows that the accuracy and robustness of this approach is on par with the state-of-the-art practices.

show abstract

“…In contrast to the single-image method, the utilization of multiple sensors enables capturing the full 3D geometry of an object [ 6 , 7 , 8 , 9 ]. By fusing the surfaces observed by multiple cameras, the 3D object can be accurately reconstructed with fewer artifacts.…”

Section: Introductionmentioning

confidence: 99%

Multiple Sensor Synchronization with theRealSense RGB-D Camera

Yoon

Jang

Huh

et al. 2021

Sensors

View full text Add to dashboard Cite

When reconstructing a 3D object, it is difficult to obtain accurate 3D geometric information using a single camera. In order to capture detailed geometric information of a 3D object, it is inevitable to increase the number of cameras to capture the object. However, cameras need to be synchronized in order to simultaneously capture frames. If cameras are incorrectly synchronized, many artifacts are produced in the reconstructed 3D object. The RealSense RGB-D camera, which is commonly used for obtaining geometric information of a 3D object, provides synchronization modes to mitigate synchronization errors. However, the synchronization modes provided by theRealSense cameras can only sync depth cameras and have limitations in the number of cameras that can be synchronized using a single host due to the hardware issue of stable data transmission. Therefore, in this paper, we propose a novel synchronization method that synchronizes an arbitrary number of RealSense cameras by adjusting the number of hosts to support stable data transmission. Our method establishes a master–slave architecture in order to synchronize the system clocks of the hosts. While synchronizing the system clocks, delays that resulted from the process of synchronization were estimated so that the difference between the system clocks could be minimized. Through synchronization of the system clocks, cameras connected to the different hosts can be synchronized based on the timestamp of the data received by the hosts. Thus, our method synchronizes theRealSense cameras to simultaneously capture accurate 3D information of an object at a constant frame rate without dropping it.

show abstract

Semantically Synchronizing Multiple-Camera Systems with Human Pose Estimation

Cited by 6 publications

References 33 publications

Effect of Timing Error: A Case Study of Navigation Camera

Effect of Timing Error: A Case Study of Navigation Camera

Spatio-Temporal Calibration of Multiple Kinect Cameras Using 3D Human Pose

Multiple Sensor Synchronization with theRealSense RGB-D Camera

Contact Info

Product

Resources

About