We demonstrate the first application of deep reinforcement learning to autonomous driving. From randomly initialised parameters, our model is able to learn a policy for lane following in a handful of training episodes using a single monocular image as input. We provide a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control. We use a continuous, modelfree deep reinforcement learning algorithm, with all exploration and optimisation performed on-vehicle. This demonstrates a new framework for autonomous driving which moves away from reliance on defined logical rules, mapping, and direct supervision. We discuss the challenges and opportunities to scale this approach to a broader range of autonomous driving tasks.

No abstract

No abstract

Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.

We design, optimize and demonstrate the behavior of a tendon-driven robotic gripper performing parallel, enveloping and fingertip grasps. The gripper consists of two fingers, each with two links, and is actuated using a single active tendon. During unobstructed closing, the distal links remain parallel, for parallel grasps. If the proximal links are stopped by contact with an object, the distal links start flexing, creating a stable enveloping grasp. We optimize the route of the active flexor tendon and the route and stiffness of a passive extensor tendon in order to achieve this behavior. We show how the resulting gripper can also execute fingertip grasps for picking up small objects off a flat surface, using contact with the surface to its advantage through passive adaptation. Finally, we introduce a method for optimizing the dimensions of the links in order to achieve enveloping grasps of a large range of objects, and apply it to a set of common household objects.