Augmenting Self-Driving with Remote Control: Challenges and Directions

Self-driving or autonomous vehicle systems are being designed over the world with increasing success in recent years. In spite of many advances so far, it is unlikely that such systems are going to ever achieve perfect accuracy under all conditions. In particular, occasional failures are anticipated when such vehicles encounter situations not observed before, or conflicting information is available to the system from the environment. Under such infrequent failure scenarios, the research community has so far, considered two alternatives to return control to the driver in the vehicle, which has its own challenges and limitations, or to attempt to safely "park" the vehicle out of harm's way. In this paper, we argue that a viable third alternative exists - on failure of the self-driving function in the vehicle, the system could return control to a remote human driver located in response centers distributed across the world. This remote human driver will augment the self-driving system in vehicles, only when failures occur, which may be due to bad weather, malfunction, contradiction in sensory inputs, and other such conditions. Of course, a remote driving extension is fraught with many challenges, including the need for some Quality of Service guarantees, both in latency and throughput, in connectivity between the vehicles on the road and the response center, so that the remote drivers can react efficiently to the road conditions. To understand some of the challenges, we have set up real-time streaming testbed and evaluate frame latency with different parameter settings under today's LTE and Wi-Fi networks. While additional optimization techniques can be applied to further reduce streaming latency, we recognize that significant new design of the communication infrastructure is both necessary and possible.