ROSEFusion

Abstract

Online reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions (<1m/s). Under very fast camera motion (e.g., 3m/s), the reconstruction can easily crumble even for the state-of-the-art methods. Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur. We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization, in particular, the Particle Filter Optimization (PFO). To surmount the computation-intensive particle sampling and update in standard PFO, we propose to accelerate the randomized search via updating a particle swarm template (PST). PST is a set of particles pre-sampled uniformly within the unit sphere in the 6D space of camera pose. Through moving and rescaling the pre-sampled PST guided by swarm intelligence, our method is able to drive tens of thousands of particles to locate and cover a good local optimum extremely fast and robustly. The particles, representing candidate poses, are evaluated with a fitness function defined based on depth-model conformance. Therefore, our method, being depth-only and correspondence-free, mitigates the motion blur impediment as (ToF-based) depths are often resilient to motion blur. Thanks to the efficient template-based particle set evolution and the effective fitness function, our method attains good quality pose tracking under fast camera motion (up to 4m/s) in a realtime framerate without including loop closure or global pose optimization. Through extensive evaluations on public datasets of RGB-D sequences, especially on a newly proposed benchmark of fast camera motion, we demonstrate the significant advantage of our method over the state of the arts.

Arm Movement Analysis Technology of Wushu Competition Image Based on Deep Learning

“…After detecting the spatiotemporal interest point in the video, the descriptor should be used to describe the point or region. According to the unique complexity of video data, the descriptor should be robust to the changes in video background, clutter, scale, and direction [13][14][15][16][17][18][19][20][21]. e so-called structure means that there are fixed topological constraints between the joint positions of bones in each frame; for example, the head joint is only connected with the neck joint.…”

Innovation and Digital Construction of Cultural Tourism Industry under the Background of Big Data and Internet of Things

“…It has played an active role in meeting the cultural needs of tourists [25]. Combined with human civilization and national culture, it will provide tourists with a more immersive experience [26,27]. erefore, it is necessary to conform to the trend of the times, dig deep into the characteristic cultural tourism resources, continuously combine various new science and technology, meet the characteristic and multilevel needs of mass tourism, promote the creative transformation and innovative development of national culture in tourism, and promote the cultural tourism industry [28][29][30].…”

Active scene reconstruction via self-rotation driven by optimized information theory

“…The corresponding result is that the scanning speed slows down. To solve this problem, Zhang et al [33] used a random optimization method based on point cloud particle swarm optimization to raise the upper limit of the instantaneous speed of the camera to 4m/s.…”

“…The camera is mounted on top of the robot and we set the maximum visual range of the depth camera to 5 meters. We choose ROSEFusion [33] as our reconstruction module because of the robustness to high-speed motion. The simulation system runs on a computer with an AMD R9 3900X CPU (12-core processor × 24 3.8Ghz), 64GB RAM, and NVIDIA Geforce RTX 3080Ti Graphics card Our benchmark dataset is built from the famous Indoor scene dataset Matterport 3D [35], which is the largest indoor 3D scanning data set at present.…”