CIRC: Dev: SMILE:Scan to Multi-sensorial Interactive Learning Environment

VR (Virtual Reality)-based immersive and interactive environments can be a great resource for learning and training, especially for concepts that involve safety aspects by interacting with inflammable or breakable objects. The tedious nature of developing these VR experiences continues to be a limiting factor for VR becoming mainstream. The main goal of this project is the design and development towards an eventual innovative infrastructure, SMILE (Scan to Multi-sensorial Interactive Learning Environment), focusing on: (1) nearly automated construction of VR environments that mimic real-world indoor scenes; (2) interactions with virtual objects involving multiple senses such as touch, visual, aural, and smell.

The SMILE infrastructure will undergo rigorous software testing to ensure safe interactions before being deployed to support Internet-scale collaboration among users. In the exploratory phase of this project, the team will develop a set of tools and software

modules for the following purposes: (a) Realistic construction of VR environments to represent real-world space(s) as closely as possible; (b) Natural interactions with the constructed VR environments using multisensorial devices; (c) Multimodal streaming to synchronize transmission and reception of multisensorial data over the Internet; (d) Testing and validating the SMILE software for robustness and safety.

Virtual chemistry laboratory experiments for assessing the performance of SMILE will be designed and developed through research collaboration with the Dallas College, an HSI (Hispanic-serving institution). Through this collaboration, the SMILE project will have a significant impact on the under-represented student community. SMILE could be used for different learning environments, apart from the Chemistry lab case study, promoting education involving K-16.

The project will involve women Ph.D. students, undergraduate, and under-represented community students.

Open-source resources produced through the SMILE project include multi-sensory database of material properties for sensorial displays, Unity-based VR authoring tools, automated software testing tools for VR-based applications. These will be made available for a period of five years through the project website at: https://labs.utdallas.edu/multimedialab/.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.