HCC: Medium: Adaptive Auditory Feedback to Improve Balance in Virtual Reality at Home

Consumer-level virtual reality goggles are not accessible for many persons with balance impairments, such as elderly persons, persons with multiple sclerosis, Parkinson's, or diabetes. Currently, people with balance impairments cannot benefit from many immersive virtual reality benefits, such as education, physical fitness, rehabilitation, and entertainment.

The team's previous work studied how audio, visuals, and vibrations can improve balance while in virtual reality in controlled laboratory settings with simplified virtual reality environments, where they found that audio was the most effective. This project aims to expand the research beyond laboratory settings because typical use of virtual reality occurs at home with complex virtual reality environments with much more intense audio, visual, and vibration stimulation.

To address the additional complexities of being outside the lab with much stronger stimulation, the team will develop specialized audio intended to improve balance in any virtual environment. If virtual reality imbalance issues can be resolved, persons with balance impairments can more readily benefit from virtual reality.

This project investigates approaches to enable adaptive auditory feedback techniques to improve balance in virtual reality use at home, which includes commercial virtual reality experiences with strong stimuli. Based on the team's preliminary studies, the central hypothesis is that adaptive auditory feedback can improve balance during virtual reality use at home more effectively than the current state of the art, which uses a static, 'one size fits all' approach to feedback.

The work will seek the following novel contributions: 1) adaptive auditory feedback techniques to improve balance in VR; 2) automatic, real-time balance prediction with low-cost sensors, some of which are already integrated into commercial virtual reality systems; 3) controlled laboratory and at-home studies, providing generalizable understanding and computational models of balance in virtual reality in the presence of strong stimuli; 4) insight into the potentially lasting effects of virtual reality-based auditory feedback on balance after virtual reality exposure. Ultimately, this project will result in datasets and open-source tools that will make virtual reality more accessible for persons with balance impairments, which could improve their quality of life.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.