Data-Driven Radar Processing Using a Parametric Convolutional Neural Network for Human Activity Classification

Abstract

The paper proposes a data-driven pre-processing optimization for radar data using a parametric convolutional neural network. The proposed method is applied on human activity classification as a use case. Present radar-based activity recognition system exploit micro-Doppler signature by generating Doppler spectrograms or a temporal series of range-Doppler maps, followed by deep neural networks or machine learning approaches for classification. Those radar data representations are typically generated on the basis of short-time Fourier transformations. A Fourier transformation equally resolves the frequency space, which may be sub-optimal in some applications. Although deep convolutional neural networks (DCNN) have been shown to implicitly learn features from raw sensor data in other fields, such as speech recognition, yet, for the case of radar-based DCNNs, pre-processing is required to develop a scalable and robust classification or regression application. In this paper, we propose a parametric convolutional neural network that mimics the radar pre-processing across fast-time and slow-time radar data through 2D sinc filter or 2D wavelet filter kernels to extract features for classification of various human activities. During training only the filter parameters of the 2D sinc filters or 2D wavelets are learned, leading to optimized feature representation for the classification task. It is demonstrated that our proposed solution shows improved results compared to equivalent DCNN architectures that rely on Doppler spectrograms or radar data cubes as input data.

Investigation for the Need of Traditional Data-Preprocessing when Applying Artificial Neural Networks to FMCW-Radar Data

“…In the specific case of radar sensors, as the ones studied in this manuscript, researchers have worked on trying to execute the radar data preprocessing using ANNs to benefit from the acceleration immanent to ANNs when using emerging devices such as Tensor Processing Units (TPU) or highpower Graphic Processing Units (GPU). Stadelmayer T. et al [5] followed this research line to develop a parametric Convolutional Neural Network (CNN) that is able to mimic the traditional preprocessing technique applied to raw radar data to extract the RDM. This CNN is based on a 2D sinc or a 2D wavelet f ilter kernel to extract features from the raw data.…”

“…These formats are often selected due to their easy feature representation in contrast with the raw radar data. However, recent works have proven it is possible to execute this preprocessing using an ANN [5]. As a result, the efficiency and accuracy of transforming radar data in contrast to the use of raw data when using ANN models is not obvious and has not been reported in the literature so far.…”

On Edge Human Action Recognition Using Radar-Based Sensing and Deep Learning

“…The last row of the table presents the proposal and includes an analysis of the two most accurate models, DNN2 and DNN3, which were trained using D Gray . All of the works achieved high accuracy in radar-based human action recognition, but some did not specifically focus on fall detection, such as [35], [36], [37], [38], and [39]. Several studies have focused on binary classification for fall detection among common daily activities, such as walking, sitting, standing, and other nonfalling actions [16], [18], [22], [40].…”

Real-Time Littering Activity Monitoring Based on Image Classification Method

“…Xinyu Li in [16] used CNN-LSTM to recognize concurrent activities. Human activity was also investigated by Thomas Stadelmayer in [17]. They used radar to help the authors record the data of daily human activities.…”

“…They claimed that the method of CNN LSTM that they proposed is robust enough to analyze the puffing with an accuracy of 78%. However, this research is almost the same as research in [17] in which it was not successful when it was run without the help of the sensors that were attached to the user's body.…”