FWNetAE: Spatial Representation Learning for Full Waveform Data Using Deep Learning

Shinohara, Takayuki; Xiu, Haoyi; Matsuoka, M.

doi:10.1109/ism46123.2019.00060

Cited by 3 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate the quality of the spatial features extracted by our FWNet, we compared the 1D CNN (shown in Section 3.3 ), FWNetAE [ 27 ], and our FWNet. To compare the power of the feature extraction, we visualized the feature vectors of the test data we extracted from the trained 1D CNN, FWNetAE, and FWNet.…”

Section: Resultsmentioning

confidence: 99%

“…The feature vectors used in the visualization were the bottleneck layer of the trained encoder. The features were extracted by FWNetAE [ 27 ] and trained with the same dataset, but the label information was omitted. FWNetAE shows the tendency to separate each class in a latent vector without supervised learning ( Figure 10 b).…”

Section: Resultsmentioning

confidence: 99%

“…However, Zorzi et al [ 26 ] demonstrated the necessity of using the spatial learning method for full-waveform lidar data via 2D FCN. Moreover, Shinohara et al (2019) showed the effectiveness of applying the spatial learning method to full-waveform lidar data using a full-waveform network auto encoder (FWNetAE), consisting of a PointNet [ 7 ] based encoder and a naïve multi-layer-perceptron based decoder [ 27 ]. FWNetAE [ 27 ] employs PointNet [ 7 ] in its encoder, which can input even spatially irregular data and directly handle waveforms and coordinates (the first peak return) associated with the waveforms, despite the fact that deep learning methods are usually used for regularly arranged data, such as images, audio, and text.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we show the effectiveness of spatial feature extraction in the semantic segmentation task for waveforms in a supervised manner. Our network is based on FWNetAE [ 27 ] using one of the typical architectures that can directly apply waveforms to representation learning tasks. We extended FWNetAE [ 27 ], where only unsupervised representation learning was possible, to directly predict the classes of waveforms in a supervised manner.…”

Section: Introductionmentioning

confidence: 99%

“…Our network is based on FWNetAE [ 27 ] using one of the typical architectures that can directly apply waveforms to representation learning tasks. We extended FWNetAE [ 27 ], where only unsupervised representation learning was possible, to directly predict the classes of waveforms in a supervised manner. Specifically, our model, namely the full-waveform network (FWNet), takes spatially distributed waveforms with associated geographical coordinates as input and predicts class vectors for each input in an end-to-end manner ( Figure 1 ).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

Shinohara

Xiu

Matsuoka

2020

Sensors

Self Cite

View full text Add to dashboard Cite

In the computer vision field, many 3D deep learning models that directly manage 3D point clouds (proposed after PointNet) have been published. Moreover, deep learning-based-techniques have demonstrated state-of-the-art performance for supervised learning tasks on 3D point cloud data, such as classification and segmentation tasks for open datasets in competitions. Furthermore, many researchers have attempted to apply these deep learning-based techniques to 3D point clouds observed by aerial laser scanners (ALSs). However, most of these studies were developed for 3D point clouds without radiometric information. In this paper, we investigate the possibility of using a deep learning method to solve the semantic segmentation task of airborne full-waveform light detection and ranging (lidar) data that consists of geometric information and radiometric waveform data. Thus, we propose a data-driven semantic segmentation model called the full-waveform network (FWNet), which handles the waveform of full-waveform lidar data without any conversion process, such as projection onto a 2D grid or calculating handcrafted features. Our FWNet is based on a PointNet-based architecture, which can extract the local and global features of each input waveform data, along with its corresponding geographical coordinates. Subsequently, the classifier consists of 1D convolutional operational layers, which predict the class vector corresponding to the input waveform from the extracted local and global features. Our trained FWNet achieved higher scores in its recall, precision, and F1 score for unseen test data—higher scores than those of previously proposed methods in full-waveform lidar data analysis domain. Specifically, our FWNet achieved a mean recall of 0.73, a mean precision of 0.81, and a mean F1 score of 0.76. We further performed an ablation study, that is assessing the effectiveness of our proposed method, of the above-mentioned metric. Moreover, we investigated the effectiveness of our PointNet based local and global feature extraction method using the visualization of the feature vector. In this way, we have shown that our network for local and global feature extraction allows training with semantic segmentation without requiring expert knowledge on full-waveform lidar data or translation into 2D images or voxels.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

Shinohara

Xiu

Matsuoka

2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

Point2Wave: 3-D Point Cloud to Waveform Translation Using a Conditional Generative Adversarial Network With Dual Discriminators

Shinohara¹,

Xiu²,

Matsuoka³

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Semantic Segmentation for Full-Waveform LiDAR Data Using Local and Hierarchical Global Feature Extraction

Shinohara

Xiu

Matsuoka

2020

Proceedings of the 28th International Conference on Advances in Geographic Information Systems

Self Cite

View full text Add to dashboard Cite

During the last few years, in the field of computer vision, sophisticated deep learning methods have been developed to accomplish semantic segmentation tasks of 3D point cloud data. Additionally, many researchers have extended the applicability of these methods, such as PointNet or PointNet++, beyond semantic segmentation tasks of indoor scene data to large-scale outdoor scene data observed using airborne laser scanning systems equipped with light detection and ranging (LiDAR) technology. Most extant studies have only investigated geometric information (x, y, and z or longitude, latitude, and height) and have omitted rich radiometric information. Therefore, we aim to extend the applicability of deep learning-based model from the geometric data into radiometric data acquired with airborne full-waveform LiDAR without converting the waveform into 2D images or 3D voxels. We simultaneously train two models: a local module for local feature extraction and a global module for acquiring wide receptive fields for the waveform. Furthermore, our proposed model is based on waveform-aware convolutional techniques. We evaluate the effectiveness of the proposed method using benchmark large-scale outdoor scene data. By integrating the two outputs from the local module and the global module, our proposed model had achieved higher mean recall value 0.92 than previous methods and higher F1 scores for all six classes than the other 3D Deep Learning method. Therefore, our proposed network consisting of the local and global module successfully resolves the semantic segmentation task of full-waveform LiDAR data without requiring expert knowledge. CCS CONCEPTS • Computing methodologies → Scene understanding.

show abstract

FWNetAE: Spatial Representation Learning for Full Waveform Data Using Deep Learning

Cited by 3 publications

References 26 publications

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

Point2Wave: 3-D Point Cloud to Waveform Translation Using a Conditional Generative Adversarial Network With Dual Discriminators

Semantic Segmentation for Full-Waveform LiDAR Data Using Local and Hierarchical Global Feature Extraction

Contact Info

Product

Resources

About