An active multimodal sensing platform for remote voice detection

Abstract

Abstract-To improve the performance and the efficiency of Laser Doppler Vibrometers (LDVs) for long-range hearing, we design an active multimodal sensing platform that integrates a Pan-Tilt-Zoom (PTZ) camera, a mirror and a Pan-Tilt-Unit (PTU) to the LDV. With the assistance of the vision and active control components, the LDV can automatically select the best reflective surfaces, point the laser beam to the selected surfaces, and quickly focus the laser beam. For accomplishing these functions, distance measurement and sensor calibration methods are proposed using the triangulation between the PTZ camera and the mirrored LDV laser beam. Based on both the measured distances and the return signal levels of the LDV, a fast and automatic LDV focusing algorithm is designed. Furthermore, strategies of surface selection and laser pointing are designed for the platform to automatically point the laser beam to the designated surfaces. Experimental results are shown to validate the performance improvement of the LDV in remote automatic voice detection by using the active multimodal sensing platform.

Active stereo vision for improving long range hearing using a Laser Doppler Vibrometer

“…Third, the laser spot is moved to the new location, and the master PTZ camera is rotated synchronously to put the laser spot in the center of images. Fourth, the laser beam of the LDV is automatically and rapidly focused based on estimated distance and the signal levels, as we did in [10]. As a summary, once the estimated distance is obtained, we search a range of focus steps of the LDV and select the focus step yield the strongest returning signal.…”

“…For example, using a COTS Polytec LDV (OFV-505), it takes about 15 seconds for the laser focusing on a surface. In our previous work [10], a vision-aided LDV system is proposed to improve the performance and the efficiency of the LDV for automatic remote hearing. The platform integrated a PTZ camera, a mirror, and a PTU to the LDV.…”

A System Approach to Adaptive Multi-Modal Sensor Designs

“…Under the principle of the system approach and performance-driven sensing, a real multimodal human signature detection sensing design is proposed and a test prototype is developed to capture visual, audio and range information at a large distance (Qu, et al, 2009;Qu, et al, 2010. The core functionality of the system is remote hearing using a unique optical sensor -Laser Doppler Vibrometer (LDV).…”

Application of a laser Doppler vibrometer for air-water to subsurface signature detection

“…For example, parabolic microphones, which can capture voice signals at a fairly large distance in the direction pointed by the microphone, could be used for remote hearing and surveillance. But it is limited due to the sensitivity of the device to noise caused by the wind or sensor motion, which has a severe effect on the signal to noise ratio [1]. Therefore, the employment of an LDV alongside a hydrophone as a type of optical-acoustic sensor for remote signal detection was conducted.…”

“…Introducing optical to acoustic; such as audio-visual, a whole host and wide range of applications are introduced such as video and audio surveillance and tracking, speech recognition, underwater sound monitoring, and UAV remote sensing and detection. However, in these systems, the acoustic sensors are typically microphones that need to be placed close to the subjects of interests and fixed on pre-determined locations, making it difficult to track the desired subjects [1]. When an acoustic wave strikes a surface, energy is coupled into seismic motion in the subsurface of the ground.…”