Visual object detection using audio data

Abstract

Nowadays Internet of Things (IoT) and Machine Learning (ML) are growing fields. One application of these two fields is object detection, which detects semantic objects using digital images and videos of classes like humans, vehicles, buildings, etc. Visual object detection systems are very effective and accurate due to the appearance information obtained from the cameras. But they face the problem of a limited Field of View. This paper aims to tackle this issue by using audio data to localize the object. A microphone is used to estimate the angular position of the object emitting the sound. Objects currently in the Field of View of a camera are detected and tracked using optical flow, but when they go out of the Field of View, the sound emitted by the object is used by the microphone to calculate the object‘s angular position. Once the angle is calculated, the camera is rotated in that direction. This thus ensures that the object can be located even if it goes out of frame. Once the object is located through IoT devices, we use ML to identify the person‘s face.