Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Abstract

Recent research on deep learning has been applied to a diversity of fields. In particular, numerous studies have been conducted on self-driving vehicles using end-to-end approaches based on images captured by a single camera. End-to-end controls learn the output vectors of output devices directly from the input vectors of available input devices. In other words, an end-to-end approach learns not by analyzing the meaning of input vectors, but by extracting optimal output vectors based on input vectors. Generally, when end-to-end control is applied to self-driving vehicles, the steering wheel and pedals are controlled autonomously by learning from the images captured by a camera. However, high-resolution images captured from a car cannot be directly used as inputs to Convolutional Neural Networks (CNNs) owing to memory limitations; the image size needs to be efficiently reduced. Therefore, it is necessary to extract features from captured images automatically and to generate input images by merging the parts of the images that contain the extracted features. This paper proposes a learning method for end-to-end control that generates input images for CNNs by extracting road parts from input images, identifying the edges of the extracted road parts, and merging the parts of the images that contain the detected edges. In addition, a CNN model for end-to-end control is introduced. Experiments involving the Open Racing Car Simulator (TORCS), a sustainable computing environment for cars, confirmed the effectiveness of the proposed method for self-driving by comparing the accumulated difference in the angle of the steering wheel in the images generated by it with those of resized images containing the entire captured area and cropped images containing only a part of the captured area. The results showed that the proposed method reduced the accumulated difference by 0.839% and 0.850% compared to those yielded by the resized images and cropped images, respectively.

Style Transformation Method of Stage Background Images by Emotion Words of Lyrics

“…Content image refers to an image that has information such as an object or a common landscape that people can usually recognize, and style image refers to an image that has information such as color or texture that will be combined with the content image. Style transfer transfers the style based on a convolutional neural network (CNN) [10,11] and a generative adversarial network (GAN) [5][6][7]. Style transfer based on the CNN model extracts features by separating content and style in an image.…”

“…Research regarding the reflection of emotions contained in song lyrics in a stage background are scarce; however, it is possible to use research that transforms background images according to their meanings or purpose by synthesizing background images with text or images containing the meaning to be represented. There is research that partially transforms images using the content contained in text [4][5][6] and transfers the style such as color, line, and texture of image to another image [7][8][9][10][11]. The existing stage background image recommendation system recommends images for dancers, but this does not include the characteristics of the song lyrics.…”

An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms

“…Although there are the prototypes of autonomous vehicles currently tested on the regular streets, some of the challenges for the autonomous driving are not completely solved yet. Current challenges in autonomous vehicles development are sensor fusion [38,39,40,41], higher-level planning decisions [42,43,44,45,46], an end-to-end learning for autonomous driving [1,2,3,4,5,47,48,49], reinforcement learning for autonomous driving [5,50,51,52,53], and human machine interaction [54,55]. A systematic comparison of deep learning architectures used for autonomous vehicles is given in [56], a short overview of sensors and sensor fusion in autonomous vehicles is presented in [57].…”

Emerging ICT UAV applications and services: Design of surveillance UAVs

“…For example, an end-toend approach based on convolutional neural networks can be utilized to generate optimal control signals based on images captured by a UAV. 17,18 However, it is necessary to conduct further research on UAV control by considering the flight paths preferred by the pilot.…”