Robust and subject-independent driving manoeuvre anticipation through Domain-Adversarial Recurrent Neural Networks

Abstract

Through deep learning and computer vision techniques, driving manoeuvres can be predicted accurately a few seconds in advance. Even though adapting a learned model to new drivers and different vehicles is key for robust driverassistance systems, this problem has received little attention so far. This work proposes to tackle this challenge through domain adaptation, a technique closely related to transfer learning. A proof of concept for the application of a Domain-Adversarial Recurrent Neural Network (DA-RNN) to multi-modal time series driving data is presented, in which domain-invariant features are learned by maximizing the loss of an auxiliary domain classifier. Our implementation is evaluated using a leave-one-driver-out approach on individual drivers from the Brain4Cars dataset, as well as using a new dataset acquired through driving simulations, yielding an average increase in performance of 30% and 114% respectively compared to no adaptation. We also show the importance of finetuning sections of the network to optimise the extraction of domain-independent features. The results demonstrate the applicability of the approach to driverassistance systems as well as training and simulation environments.Experiment 3 Finally, we implement the same domain-adversarial approach to study how the network, trained only on the Brain4Cars data, adapts to our new dataset, in which the drivers and the driving set-up -e.g. position of the mirrors, windows, and the camera-differ from the Brain4Cars dataset.The results confirm that, without adaptation, the model is not able to predict manoeuvres from observations in which the features have very different marginal distributions compared to the training set. We conclude discussing the potential applications of the domain-adversarial approach to apply domain 4

A Survey of Unsupervised Deep Domain Adaptation

“…For time-series data, domain adaptation has been used for learning temporal latent relationships in health data across different population age groups [189], to perform speech recognition [101,217,281], for predicting driving maneuvers [237], anomaly detection [241], and inertial tracking [35]. In a method addressing the related problem of domain generalization, time-series radio data was used for sleep-stage classification [283].…”

Transfer Learning and Deep Domain Adaptation

“…Another real-world task that involves time-series data is to build diver assistant systems. In [52], an auxiliary domain classifier is also adopted to enhance the performance of recurrent neural networks for driving maneuvers anticipation. And the core idea in this paper is also to learn sharing features from different datasets by the domain classifier.…”

Advances and Applications in Deep Learning

“…For example, Li et al [51] proposed a fully convolutional localization network for extracting representation from images and the decoder for generating captions is LSTM. Recently, attention mechanism has been widely used for sequence processing and achieved significant improvements such as machine translation, Huang et al [52] introduce an encoderdecoder framework, where an attention module is used in the encoder and decoder respectively. Specifically, the encoder is a CNN based network.…”