Interactions with Future Reach-Through Volumetric Displays

Displays in the shape of televisions, computer screens or phones are ever present in our education, work and entertainment. However, they do not take full advantage of our inner spatial abilities that we have to interact with the real world. True 3D displays can provide the same visual clues as the real world without forcing the users to wear devices.

However, with State of the art (SoA) displays, the users cannot reach inside the display volume to directly interact with the virtual objects as they would do in real life.

We envision a volumetric display capable of projecting true 3D virtual objects in mid-air that can be reached by the users to enable direct interaction, i.e. a reach-through volumetric display (RVD).

This vision has been presented in multiple movies and books but there is no realization.Three novel technologies will be developed and combined to create an RVD. 1) fast time-multiplexed acoustic fields will create virtual force fields that give shape to microfabricated light-scaterring particles. 2) Tomographic illumination will shine on the particles as a more scalable alternative to phase-based holographic. 3) Volumetric tracking of the particle distribution will control the previous technologies in a closed-loop manner.

Applications will serve as benchmarks to test novel interaction techniques and develop a framework that fills in the knowledge gap for interactions with as yet nonexistent RVDs.

The objectives of the project are: O1) find a set of technologies that enables the realization of RVDs, O2) create interaction techniques for RVDs and categorize them using a framework that will be applicable to future displays, our current frameworks for 3d-interactions may not be applicable to RVDs.

O3) enhance an RVD with tactile sensations, spatial audio and study its effects on humans.

The PI is uniquely qualified with experience in designing mid-air interactions and using levitated particles for displays.