Concept detection and face pose estimation using lightweight convolutional neural networks for steering drone video shooting

Abstract

Abstract-Unmanned Aerial Vehicles, also known as drones, are becoming increasingly popular for video shooting tasks since they are capable of capturing spectacular aerial shots. Deep learning techniques, such as Convolutional Neural Networks (CNNs), can be utilized to assist various aspects of the flying and the shooting process allowing one human to operate one or more drones at once. However, using deep learning techniques on drones is not straightforward since computational power and memory constraints exist. In this work, a quantizationbased method for learning lightweight convolutional networks is proposed. The ability of the proposed approach to significantly reduce the model size and increase both the feed-forward speed and the accuracy is demonstrated on two different drone-related tasks, i.e., human concept detection and face pose estimation.

Autonomous UAV Cinematography

“…Luckily, operating in global 3D Cartesian coordinates makes it meaningful to integrate a 3D visual target pose estimation algorithm into the visiongroup pipeline. There are two main approaches to achieve this: a) the computer vision approach, where predefined landmark points are detected/tracked on the target's image and used to solve the Perspective-n-Point problem [53], or b) the machine learning approach, where the target's pose is directly regressed by a trained model that only uses the visual input [54] [55]. The first approach requires a 3D model of the target to be known, while the second solution requires a regressor properly trained on a representative, fully annotated image dataset.…”

Challenges in Autonomous UAV Cinematography: An Overview

“…The challenge lies in optimizing them for real-time performance at a low energy expenditure envelope, using the relatively limited computational hardware of UAV platforms. This is a significant issue that only recently has begun to be investigated (e.g., [37]).…”

Visual Object Detection For Autonomous UAV Cinematography

“…The use of camera-equipped Unmanned Aerial Vehicles (UAVs) for covering public sport events, such as bicycle or boat races, parkour shows and football games, as well as for media production, surveillance, search and rescue operations, etc., is becoming increasingly popular, since UAVs are capable of shooting spectacular videos that would otherwise be very difficult and costly to obtain. Visual analysis tasks may, thus, be of assistance in UAVbased intelligent cinematography [5,12,14,16], e.g., for detecting and tracking a desired target, or even in flight safety related tasks [21], such * Corresponding Author: foteinpp@csd.auth.gr as obstacle detection and avoidance. Technological progress has led to the production of numerous commercially available UAVs with similar cognitive autonomy and perceptual capabilities, but the limited computational hardware, the possibly high camera-to-target distance and the fact that both the UAV/camera and the target(s) are moving, constitute achieving both high accuracy and real-time performance rather challenging [9,10,11].…”