Real-Time 3D Reconstruction Method Based on Monocular Vision

Jia, Qingyu; Chang, Liang; Qiang, Baohua; Zhang, Shihao; Xie, Wu; Yang, Xianyi; Sun, Yangchang; Yang, Minghao

doi:10.3390/s21175909

Cited by 7 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jia et al [ 15 ] proposed a real-time 3D reconstruction method based on monocular vision. One RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment visual information to extract some of the important visual information.…”

Section: Introductionmentioning

confidence: 99%

On the Problem of Restoring and Classifying a 3D Object in Creating a Simulator of a Realistic Urban Environment

Gorodnichev

Erokhin

Polyantseva

et al. 2022

Sensors

View full text Add to dashboard Cite

Since the 20th century, a rapid process of motorization has begun. The main goal of researchers, engineers and technology companies is to increase the safety and optimality of the movement of vehicles, as well as to reduce the environmental damage caused by the automotive industry. The difficulty of managing traffic flows is that cars are driven by a person and their behavior, even in similar situations, is different and difficult to predict. To solve this problem, ground-based unmanned vehicles are increasingly being developed and implemented; however, like any other intelligent system, it is necessary to train different road scenarios. Currently, an engineer is driving an unmanned vehicle for training and thousands of kilometers are being driven for training. Of course, this approach to training unmanned vehicles is very long, and it is impossible to reproduce all the scenarios that can be found in real operations on a real road. Based on this, we offer a simulator of a realistic urban environment which allows you to reduce the training time and allows you to generate all kinds of events. To implement such a simulator, it is necessary to develop a method that would allow recreating a realistic world in one passage with cameras (monocular) installed on board the vehicle. Based on this, the purpose of this work is to develop an intelligent vehicle recognition system using convolutional neural networks, which allows you to create mesh objects for further placement in the simulator. It is important to note that the resulting objects should be optimal in size so as not to overload the system, since a large number of road infrastructure objects are stored there. Also, neural complexity should not be excessive. In this paper, the general concept and classification of convolutional neural networks are given, which allow solving the problem of recognizing 3D objects in images. Based on the analysis, the existing neural network architectures do not solve the problems mentioned above. In this connection, the authors first of all carried out the design of the system according to the methodology of modeling business processes, and also modified and developed the architecture of the neural network, which allows classifying objects with sufficient accuracy, obtaining optimized mesh objects and reducing computational complexity. The methods proposed in this paper are used in a simulator of a realistic urban environment, which reduces the time and computational costs when training unmanned transport systems.

show abstract

Section: Introductionmentioning

confidence: 99%

On the Problem of Restoring and Classifying a 3D Object in Creating a Simulator of a Realistic Urban Environment

Gorodnichev

Erokhin

Polyantseva

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Monocular depth estimation refers to the ability to learn a dense depth map at the pixel level from the video stream. It is a fundamental challenge in the field of computer vision with potential applications in robotics, autonomous driving, 3D reconstruction, and medical imaging [1][2][3][4]. How to predict a high quality dense depth map remains a problem to be solved.…”

Section: Introductionmentioning

confidence: 99%

The Constraints between Edge Depth and Uncertainty for Monocular Depth Estimation

Liang

et al. 2021

Electronics

View full text Add to dashboard Cite

The self-supervised monocular depth estimation paradigm has become an important branch of computer vision depth-estimation tasks. However, the depth estimation problem arising from object edge depth pulling or occlusion is still unsolved. The grayscale discontinuity of object edges leads to a relatively high depth uncertainty of pixels in these regions. We improve the geometric edge prediction results by taking uncertainty into account in the depth-estimation task. To this end, we explore how uncertainty affects this task and propose a new self-supervised monocular depth estimation technique based on multi-scale uncertainty. In addition, we introduce a teacher–student architecture in models and investigate the impact of different teacher networks on the depth and uncertainty results. We evaluate the performance of our paradigm in detail on the standard KITTI dataset. The experimental results show that the accuracy of our method increased from 87.7% to 88.2%, the AbsRel error rate decreased from 0.115 to 0.11, the SqRel error rate decreased from 0.903 to 0.822, and the RMSE error rate decreased from 4.863 to 4.686 compared with the benchmark Monodepth2. Our approach has a positive impact on the problem of texture replication or inaccurate object boundaries, producing sharper and smoother depth images.

show abstract

Real-Time 3D Reconstruction for Mixed Reality Telepresence Using Multiple Depth Sensors

Ishigaki

Ismail

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Real-Time 3D Reconstruction Method Based on Monocular Vision

Cited by 7 publications

References 40 publications

On the Problem of Restoring and Classifying a 3D Object in Creating a Simulator of a Realistic Urban Environment

On the Problem of Restoring and Classifying a 3D Object in Creating a Simulator of a Realistic Urban Environment

The Constraints between Edge Depth and Uncertainty for Monocular Depth Estimation

Real-Time 3D Reconstruction for Mixed Reality Telepresence Using Multiple Depth Sensors

Contact Info

Product

Resources

About