Automatic View Finding for Drone Photography based on Image Aesthetic Evaluation

Abstract

Consumer-level remotely controlled smart drones are usually equipped with high resolution cameras, which make them possible to become unmanned "flying camera". For this purpose, in this paper, we propose an automatic view finding scheme which can autonomously navigate a drone to an proper space position where a photo with an optimal composition can be taken. In this scheme, an automatic aesthetic evaluation for image composition is introduced to navigate the flying drone. It is accomplished by applying commonly used composition guidelines on the image transmitted from the drone at current view. The evaluation result is then conversely used to control the flight and provide feedback for the drone to determine its next movement. In flight control, we adopt a downhill simplex strategy to search for the optimal position and viewing direction of the drone in its flying space. When the searching converges, the drone stops and take an optimal image at current position.

AutoPhoto: Aesthetic Photo Capture using Reinforcement Learning

“…b) Rule of Thirds: aligns an object of interest on the lower-left or lower-right third of the image. Since we do not have a predefined object, at each time step we compute the salient objects of the scene (similar to [14], [16]) using saliency detection network BASNet [36]. If a salient object satisfies the rule of thirds, then CAPTURE is selected.…”

“…Further, cinematography is not strictly concerned with aesthetic view capture; instead, it is focused on capturing events or telling a story [12], [13]. As a result, some existing work simplify aesthetic estimation by leveraging heuristics like shot templates or optimizing rule of thirds with respect to actors in a scene [2], [14]- [16]. In this paper, we are interested in optimizing image aesthetics with respect to datadriven aesthetics models to better capture human preferences.…”

“…A challenge with photo viewpoint optimization with respect to a learned aesthetic function is that it is difficult to formulate analytically (contrast this with the rule of thirds which can be directly optimized given an object of interest, e.g. [14], [16]). As such, we use reinforcement learning to learn a controller that can navigate to views that are aesthetic with respect to a learned aesthetic model.…”

“…RELATED WORK a) Image Aesthetics: Understanding human judgements of image aesthetics has a long history in psychology and neuroscience, and photographers follow well-known rules to capture aesthetic images [19]. For automatic view selection, such rules can be captured by heuristics like the rule of thirds and template matching [2], [14]- [16]. Computer vision models have also been developed to estimate the aesthetics of images [19].…”

Application research of image recognition technology based on CNN in image location of environmental monitoring UAV

“…The system landed in an area of 1 m × 1 m, scanned the captured images, and detected landing marks. Xiong X [17] proposed an automatic view finding scheme that can autonomously navigate the drone to an appropriate spatial location where it can take a photo with the best composition. Oh, Jae Hong [18] used overlapping images captured by UAV to automatically reconstruct power lines in object space.…”