REU Site: Collaborative: Making Augmented and Virtual Reality Accessible

Augmented and virtual reality (XR) technologies are becoming popular consumer products. However, at present XR devices, development tools, and content pose major accessibility challenges for people with disabilities. How can you describe a 3D virtual environment to a person with a visual impairment? How should you present captions in 3D to a person who is hard-of-hearing? How can a person with a motor control disability navigate a 3D world?

This REU Site brings together Profs. Shiri Azenkot and Brian Smith, experts in accessibility, along with Prof. Steven Feiner, an augmented reality and 3D interaction pioneer.

The site will train students to be future leaders in XR accessibility. Students will receive training in accessibility and XR, engage in the research process from ideation to presentations, and join a community of scholars and practitioners who will provide career advice and mentorship. In addition, the projects undertaken by REU students will form the basis for new collaborations among the investigators and other collaborators at Columbia and Cornell.

The site will recruit students from underrepresented backgrounds, especially students with disabilities. The target is to have a mixed-ability cohort where half the students have a disability and half do not. Students will participate in a range of formal and informal learning, including an XR development "bootcamp," conversations with XR experts across academia and industry, and presentations on their work progress.

Students will work on projects at the intersection of accessibility and XR. Examples of possible projects include designing descriptions that provide overviews of virtual environments for people who are blind and designing effective navigation techniques in a virtual environment, also for people who are blind.

The broader impacts of this proposal are two-fold. First, it supports the NSF’s mission of broadening participation by focusing recruitment efforts on students from underrepresented groups, especially students with disabilities. Second, the focus of the research will benefit people with disabilities around the world.

As XR becomes an increasingly popular consumer platform, the REU research will help ensure that it provides an equal user experience to people regardless of their disability status.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.