The advent of machine learning (ML) methods for the industry has opened new possibilities in the automotive domain, especially for Advanced Driver Assistance Systems (ADAS). These methods mainly focus on specific problems ranging from traffic sign and light recognition to pedestrian detection. In most cases, the computational resources and power budget found in ADAS systems are constrained while most machine learning methods are computationally intensive. The usual solution consists in adapting the ML models to comply with the memory and real-time (RT) requirements for inference. Some models are easily adapted to resource-constrained hardware, such as Support Vector Machines, while others, like Neural Networks, need more complex processes to fit into the desired hardware. The ADAS hardware (HW platforms) are diverse, from complex MPSoC CPUs down to classical MCUs, DPSs and application-specific FPGAs and ASICs or specific GPU platforms (such as the NVIDIA families Tegra or Jetson). Therefore, there is a tradeoff between the complexity of the ML model implemented and the selected platform that impacts the performance metrics: function results, energy consumption and speed (latency and throughput). In this paper, a survey in the form of systematic review is conducted to analyze the scope of the published research works that embed ML models into resource-constrained implementations for ADAS applications and what are the achievements regarding the ML performance, energy and speed trade-off. INDEX TERMS Machine learning, embedded software, automotive engineering, GPU, FPGA, ADAS. I. INTRODUCTION Safety is increasingly important for drivers and other users of the driving environment such as pedestrians, cyclists, bikers, scooters, etc. Only in 2016, road accidents in the European Union summed up to 25.600 fatalities and 1.4 million people injured [1]. Many measures are being taken by public institutions (improving road quality, enhancing driver consciousness and driving regulations, etc.) to improve these numbers. However, the cause of most of these accidents resides in human errors or distractions. For this reason, automotive firms are pushing up research on improving safety. The cornerstone of these advances are Advanced Driver Assistance Systems (ADAS). ADAS systems consist in several sensors, processing and actuators components that help drivers to avoid accidents and The associate editor coordinating the review of this manuscript and approving it for publication was Ye Duan.

No abstract

On the road to making self-driving cars a reality, academic and industrial researchers are working hard to continue to increase safety while meeting technical and regulatory constraints Understanding the surrounding environment is a fundamental task in self-driving cars. It requires combining complex computer vision algorithms. Although state-of-the-art algorithms achieve good accuracy, their implementations often require powerful computing platforms with high power consumption. In some cases, the processing speed does not meet real-time constraints. FPGA platforms are often used to implement a category of latency-critical algorithms that demand maximum performance and energy efficiency. Since self-driving car computer vision functions fall into this category, one could expect to see a wide adoption of FPGAs in autonomous cars. In this paper, we survey the computer vision FPGA-based works from the literature targeting automotive applications over the last decade. Based on the survey, we identify the strengths and weaknesses of FPGAs in this domain and future research opportunities and challenges.INDEX TERMS autonomous car, feature extraction, field programmable gate arrays, image classification, object detection, reconfigurable architectures Self-driving functions can be implemented in several computing platforms such as ASICs, CPUs, DSPs, GPUs, FPGAs, or in heterogeneous platforms composed by any

No abstract

No abstract

Recovering a high dynamic range (HDR) image from a single low dynamic range (LDR) image, namely inverse tone mapping (ITM), is challenging due to the lack of information in over-and under-exposed regions. Current methods focus exclusively on training high performing but computationally inefficient ITM models, which in turn hinder deployment of the ITM models in resource-constrained environments with limited computing power such as edge and mobile device applications.To this end, we propose combining efficient operations of deep neural networks with a novel mixed quantization scheme to construct a well performing but computationally efficient mixed quantization network (MQN) which can perform single image ITM on mobile platforms. In the ablation studies, we explore the effect of using different attention mechanisms, quantization schemes and loss functions on the performance of MQN in ITM tasks. In the comparative analyses, ITM models trained using MQN perform on par with the state-of-the-art methods on benchmark datasets. MQN models provide up to 10 times improvement on latency and 25 times improvement on memory consumption.

No abstract

Vision-based bronchoscopy (VB) models require the registration of the virtual lung model with the frames from the videobronchoscopy to provide an effective guidance during the biopsy. The registration can be achieved by either tracking the position and orientation of the bronchoscopy camera, or by calibrating its deviation from the pose (position and orientation) simulated in the virtual lung model. Recent advances in neural networks and temporal image processing have provided new opportunities for guided bronchoscopy. However, such progress has been hindered by the lack of comparative experimental conditions.In the present paper we share a novel synthetic dataset allowing for a fair comparison of methods. Moreover, this paper investigates several neural network architectures for the learning of temporal information at different levels of subject personalization. In order to improve orientation measurement, we also present a standartized comparison framework and a novel metric for camera orientation learning. Results on the dataset show that the proposed metric and architectures, as well as the standardized conditions, provide notable improvements to current state of the art camera pose estimation in videobronchoscopy.