Non-Contact Vibro-Acoustic Object Recognition Using Laser Doppler Vibrometry and Convolutional Neural Networks

Abstract

Laser Doppler vibrometers (LDVs) have been widely adopted due to their large number of benefits in comparison to traditional contacting vibration transducers. Their high sensitivity, among other unique characteristics, has also led to their use as optical microphones, where the measurement of object vibration in the vicinity of a sound source can act as a microphone. Recent work enabling full correction of LDV measurement in the presence of sensor head vibration unlocks new potential applications, including integration within autonomous vehicles (AVs). In this paper, the common AV challenge of object classification is addressed by presenting and evaluating a novel, non-contact vibro-acoustic object recognition technique. This technique utilises a custom set-up involving a synchronised loudspeaker and scanning LDV to simultaneously remotely solicit and record responses to a periodic chirp excitation in various objects. The 864 recorded signals per object were pre-processed into spectrograms of various forms, which were used to train a ResNet-18 neural network via transfer learning to accurately recognise the objects based only on their vibro-acoustic characteristics. A five-fold cross-validation optimisation approach is described, through which the effects of data set size and pre-processing type on classification accuracy are assessed. A further assessment of the ability of the CNN to classify never-before-seen objects belonging to groups of similar objects on which it has been trained is then described. In both scenarios, the CNN was able to obtain excellent classification accuracy of over 99.7%. The work described here demonstrates the significant promise of such an approach as a viable non-contact object recognition technique suitable for various machine automation tasks, for example, defect detection in production lines or even loose rock identification in underground mines.

A Systematic Review of Advanced Sensor Technologies for Non-Destructive Testing and Structural Health Monitoring

“…LDVs measure vibrations without mass-loading targets, which is especially important for MEMS devices. Darwish et al [ 387 ] used LVD and convolutional neural networks for non-contact vibroacoustic object recognition.…”

Application of Time Synchronous Averaging in Mitigating UAV Noise and Signal Loss for Continuous Scanning Laser Doppler Vibrometry

“…Another challenge in LDV-mounted UAV solutions arises from the necessity of a tracking system to precisely target specific points on a structure for vibration measurement [12]. Even minor misalignments can lead to signal loss or flawed data, a critical concern when the UAV is subjected to dynamic environmental factors such as wind or turbulence [13].…”

Recurrence Rate spectrograms for the classification of nonlinear and noisy signals

“…Often important signal figures are neglected or are assumed as noise, especially when only analyzing the Fourier transform {i} spectrum. In recent years, Machine Learning (ML) based techniques using Convolutional Neural Networks (CNNs) {ii} and their application to classifying acoustic or vibration data using periodogram techniques have found growing popularity [9][10][11].…”