BLAINDER—A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data

Abstract

Common Machine-Learning (ML) approaches for scene classification require a large amount of training data. However, for classification of depth sensor data, in contrast to image data, relatively few databases are publicly available and manual generation of semantically labeled 3D point clouds is an even more time-consuming task. To simplify the training data generation process for a wide range of domains, we have developed the BLAINDER add-on package for the open-source 3D modeling software Blender, which enables a largely automated generation of semantically annotated point-cloud data in virtual 3D environments. In this paper, we focus on classical depth-sensing techniques Light Detection and Ranging (LiDAR) and Sound Navigation and Ranging (Sonar). Within the BLAINDER add-on, different depth sensors can be loaded from presets, customized sensors can be implemented and different environmental conditions (e.g., influence of rain, dust) can be simulated. The semantically labeled data can be exported to various 2D and 3D formats and are thus optimized for different ML applications and visualizations. In addition, semantically labeled images can be exported using the rendering functionalities of Blender.

GPU Rasterization-Based 3D LiDAR Simulation for Deep Learning

“…Subsequently, an enhanced version was developed in C++, which led to a substantial reduction in simulation time, as discussed in [43]. More recently, the latest generic ToF simulator to emerge is Blainder [18], a simulator integrated with Blender [44]. Blainder offers the capability to generate meticulously annotated synthetic point data, marking a significant advancement in ToF simulation technology.…”

“…Given the close correspondence between the simulation process and the physical ToF device, it is unsurprising that a significant portion of ToF simulators rely on such ray casting methodologies. As far as our knowledge extends, all generic ToF simulators adopt a raycasting-based approach, including examples like Helios [42], its improved version [43], and Blainder [18]. These simulators are strictly implemented on the CPU and often cast rays either sequentially or with limited parallelization.…”

“…Though focusing on AI for solid-state LiDAR, we introduce concepts that are generic and can provide means to enable deep-learning on other sensing devices for which no annotated data are available. From a technical perspective, the proposed simulator harvests the computational power of the GPU by exploiting the real-time graphics rendering pipeline, hence significantly increasing data generating speeds compared to the state-of-the-art ToF simulators, which rely on ray casting [18,19]. Experiments reveal that the proposed simulator mimics reality accurately and is suitable for generating datasets to train neural networks.…”

Developing digital twins of multi-camera metrology systems in Blender

“…This provides a potential environment for the automated exploration of design choices when creating a camera-based 3D measurement system. Recent digital applications using Blender relevant to metrology have been developed, such as BLAINDER [35], an add-on for Blender allowing different depth sensors to be loaded from pre-sets; customised sensors can be implemented and different environmental conditions (e.g. influence of rain, dust) can be simulated, and BlenderProc [36], a procedural pipeline helping in generating realistic looking images for the training of convolutional neural networks.…”

Using Blender as a Tool to Simulate 3d Camera Based Measurements

“…In this way it is interesting for the automated exploration of design choices when creating a camera-based 3D measurement system. Recent digital applications relevant to metrology have been developed using Blender, such as BALINDER [1] and BlenderProc [2]. However, as generally Blender is not typically used for metrology simulation applications, there are no evaluations of Blender for this purpose.…”