Subject identification has several applications. In transportation companies, knowing who is driving their vehicles might prevent theft or other ill-intended actions. On the other hand, privacy concerns, and the respective legislation, hinder the applicability of several traditional recognition techniques based on invasive technologies, such as video cameras. Moreover, in order to keep the driver's distractions to a minimum, this technologies must be non-disruptive, that is, they must be able to identify the subject seamlessly without them taking any action. In this context, we propose using deep learning applied to smart watch data for recognizing the person driving a vehicle based on a training set. Our proposal relies on the possibility of using transfer learning to avoid long training sessions for new drivers and to deliver a solution which can be deployed in practice. In this paper, we describe the convolutional neural network used in the solution and present results according to a real data-set collected by us, achieving accuracies ranging from 75 to 94%.