IRES Track 1: International Research Experiences in Learning based Connected and Autonomous Vehicles (CAVs) with Real-World Implementations

Future societies will depend more and more on artificial intelligence (AI) and networked systems, and, in turn, on autonomous vehicles (AVs) and connected AVs (CAVs) for many services and operations; i.e. transportation, urban logistics, factory automation, smart farming/agriculture and disaster management, to name a few. AVs and CAVs have strong potential to increase performance, safety, and efficiency, and as a result, to contribute to societal well-being and enhance economic growth.

On the other hand, autonomous systems still fail to provide generalizable responses to vehicle, sensor, road, and environment related uncertainties, and need human intervention. Reinforcement learning (RL), an emerging branch of machine learning (ML) and control, has a lot to promise for autonomy with its capacity to address unpredictable changes in the system and environment.

However, the field still has many research gaps and also suffers from the lack of practical research evaluation. Most AI based AV research is performed in simulations, on simple platforms and for simplified cases that are far from reflecting real-life uncertainties and convincing responses to changing road conditions, especially at high speeds. This 3-year project will support selected undergraduate and graduate students from US universities to tackle the open challenges of fully autonomous vehicles within a cohort experience at Istanbul Technical University (ITU) under the mentorship of subject-matter experts from ITU, from KTH, and a US based autonomous bus company (ADASTEC Corp).

Eight US students (4 graduate, 4 undergraduate) will be funded each year for a 10-week on-site, hands-on research experience at ITU, with each student being in charge of his/her own level-appropriate project using ML/RL for one or more of the vehicle autonomy layers; namely, for perception, localization, motion planning, and trajectory tracking and associated practical tests on actual vehicles (with safety drivers). The algorithm tests will be performed around the ITU Campus within real-world scenarios.

Each year, a different student cohort will be selected for this unique research and professional development opportunity, thereby contributing to the US leadership in the future of vehicle technologies with a well-prepared workforce. Special recruitment efforts are planned for broadening participation and recruitment of students from underrepresented communities.

The adaptive optimality offered by RL provides increased performance and efficiency when compared with classical control approaches that cannot handle unstructured dynamics, often resulting in safe but highly conservative, low-performance solutions. Similarly, because of its adaptability to changes in the environment and problem dynamics, RL offers increased safety when compared with rule-based, heuristic approaches often practiced in industries to address the needs of autonomous vehicles and platforms.

The student research projects for each layer of vehicle autonomy will use RL based designs to address uncertainties and disturbances faced in real-life, which have often been ignored in lab based AV research. In our perception/localization projects, novel algorithms will be developed to estimate sensing uncertainties to improve the performance of RL based motion planning and dynamic object tracking.

The trajectory tracking algorithms will be based on Zero-Sum Games (ZSG) and online, model-free RL based control algorithms to address disturbances and slip/slide effects, which is also a significant research contribution. At the end of each research cycle, the modularly developed algorithms will be integrated on an actual vehicle and tested individually and in integration, first on the ITU vehicle-in-the-loop (VIL) system, then on indoor RaceCars and finally, at the ITU Living Lab environment for shuttle service scenarios.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.