CAREER: Enhancing Energy Efficiency in Mobile Augmented Reality Apps

Mobile Augmented Reality (MAR) apps leverage the camera of mobile devices to augment the real user environment with a set of virtual objects. Users can thus achieve a high level of engagement by interacting with the augmented environment through the mobile user interface. Typical MAR apps require executing several compute-intensive tasks to render virtual objects and, using Artificial Intelligence (AI), analyze the environment.

However, due to their relatively small form factor, mobile devices only have limited computing resources and energy available for the MAR app execution, which may lead to short battery life and poor user experience. To ensure the future success of MAR, it is thus important not only to conduct novel research on how to enable high energy efficiency in MAR apps but also to educate the future workforce in the best practices for energy efficient mobile software design.

To achieve these goals, this project will propose an edge-assisted and energy-efficient framework for MAR apps. Specifically, the project designs runtime algorithms that (1), on each mobile device, control user-perceived performance and energy consumption by manipulating virtual object quality, AI-model complexity, and AI-task allocation on the heterogeneous computing resources, (2), on the supporting edge servers, ensure scalability by grouping similar virtual object decimation requests for minimized cross-user performance cost under limited edge computing and storage resources, and (3) will be implemented and tested on a physical testbed.

By providing a framework that makes MAR apps more energy-efficient, this project will help speed up the adoption of augmented reality in many different key sectors of modern society, such as education and workforce training, and will unlock the full potential of new technologies such as augmented remote meetings and augmented experiences in public places. The project will also result in the creation of new graduate-level courses to educate the future workforce on energy-efficient mobile resource management, new research opportunities for undergraduate students, and new summer camps for K12 students.

The course material will be made available to other universities interested to introduce similar courses, projects, and summer camps. The results of this project will be disseminated through publications in top-tier conferences and journals. Similarly, all the source code of the proposed algorithms will be made available as open source.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.