A High-Speed Imaging Method Based on Compressive Sensing for Sound Extraction Using a Low-Speed Camera

Abstract

This paper reports an efficient method for sound extraction from high-speed light spot videos reconstructed from the coded light spot images captured with a low-speed camera based on compressive sensing, but at the expense of consuming time. The proposed method first gets the high-speed video of the light spot that is illuminated on the vibrating target caused by sound. Then the centroid of the light spot is used to recover the sound. Simulations of the proposed method are carried out and experimental results are demonstrated. The results show that high-speed videos with a frame rate of 2000 Hz can be reconstructed with a low-speed (100 Hz) charge-coupled device (CCD) camera, which is randomly modulated by a digital micro-mirror device (DMD) 20 times during each exposure time. This means a speed improvement of 20 times is achieved. The effects of synchronization between CCD image recording and DMD modulation, the optimal sampling patterns of DMD, and sound vibration amplitudes on the performance of the proposed method are evaluated. Using this compressive camera, speech (counting from one to four in Chinese) was recovered well. This has been confirmed by directly listening to the recovered sound, and the intelligibility value (0–1) that evaluated the similarity between them was 0.8185. Although we use this compressive camera for sound detection, we expect it to be useful in applications related to vibration and motion.

A Speech Enhancement Algorithm for Speech Reconstruction Based on Laser Speckle Images

“…Regarding speech reconstruction algorithm, researchers have proposed a variety of methods, including digital image correlation (DIC) [ 19 , 20 , 21 , 22 , 23 , 24 ], optical flow method [ 16 , 25 , 26 ], and intensity method [ 27 , 28 ]. Ref.…”

Simulating and Testing Microvibrations on an Optical Satellite Using Acceleration Sensor-Based Jitter Measurements

“…The narrow line was then extracted from the image, and the strip noise was removed by a smoothing filter. The centroid of each column was calculated using arithmetic mean [31,32,33]. The pixel offset in the time domain could, thus, be obtained.…”

Swept coded aperture real-time femtophotography

“…This pixel-wise coded exposure was implemented by using various techniques, such as spatial light modulators (e.g., a digital micromirror device 35 , 36 and a liquid-crystal-on-silicon device 37 ), a translating printed pattern 38 , 39 , and in-pixel memory in the CMOS architecture 40 . However, the imaging speeds enabled by these methods are clamped to several thousand fps by either the pattern refreshing rates of the spatial light modulators 41 , the moving speed of the piezo stages, or the readout electronics of the imaging sensor. Although CUP provides an ultrafast pixel-wise encoding scheme, its operating principle requires simultaneously shearing the scene and the coded aperture.…”