EAGER: Exploring Multi-Modal Deep Learning Systems for Sustainable Connected and Autonomous Vehicles

Autonomous vehicles have received a lot of attention in recent years, with experimental cars from Waymo, Uber, Tesla, and others being tested on roads. Such vehicles have the potential to eliminate human errors (including distracted driving) that are the cause of more than 90% of all road accidents. The benefits extend beyond safety, e.g., the adoption of self-driving vehicles on U.S. roadways is expected to reduce greenhouse emissions by 87–94% per vehicle by 2030.

However, well-publicized recent fatalities and accidents involving self-driving vehicles point to key challenges that remain unaddressed. Recently, connected autonomous vehicles (CAVs) have emerged, with the potential to improve self-driving vehicle safety and fuel economy, by communicating with other vehicles and infrastructure to share information about road hazards, pedestrians, etc.

But CAV safety and sustainable operation assurances remain elusive, due to their significantly greater complexity compared to the most advanced vehicles on the roads today. This EAGER proposal will perform critical early exploratory research to lay the foundations of robust sensing, communication, localization, security, and control in CAVs, to enable end-to-end guarantees for real-time safety and sustainable fuel economy.

The proposed research will study the susceptibility of state-of-the-art deep machine learning algorithms for sensing, scheduling, localization, anomaly detection, and energy-optimal control to uncertainties from adversarial attacks, sensor faults, timing aberrations, and other sources. For the first time, the impact of uncertainties across individual vehicular subsystems will be quantified on the security, fuel economy, driving performance, and emergent behaviors of the overall CAV system.

This exploratory analysis will allow for the realization of powerful new countermeasures to improve uncertainty robustness, predictability, and performance in the “system of deep learning systems” responsible for making decisions in a CAV.

By quantifying and overcoming varied and dynamic sources of uncertainty in emerging connected and autonomous vehicles, this project will usher in more robust, safe, and sustainable self-driving vehicles. This outcome will transform ground transportation and modern society, laying the groundwork to eliminate thousands of fatalities on U.S. roads, improving fuel economy, enhancing comfort during transportation, and saving the U.S. economy billions of dollars annually in lost productivity due to accidents and traffic congestion.

The research thrusts are foundational and can be applied to a broad range of applications, wherever the emphasis is on creating uncertainty-resilient multi-agent systems, e.g., swarms of unmanned aerial or underwater vehicles. All vehicle drive-cycle datasets and algorithms from the project will be open sourced to further research in the emerging interdisciplinary area of autonomous vehicle safety and sustainability.

Research efforts will also be tightly integrated with outreach efforts to include women, underserved, graduate, undergraduate, and K-12 students in research that has a highly positive impact on society.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.