Frameworks of transdisciplinary practice for the creation of interactive and immersive XR experiences, incorporating generative AI and real-time data

This research project offers an investigation into the application of Generative Artificial Intelligence (AI) Digital Entities in Mixed Reality (XR) with the aim to augment user engagement and facilitate the tailoring of XR environments and experiences through the integration of real-time data. It is posited that a digital entity, deliberately designed to be contextually aware through the interpretation of real-time data, possesses the potential to enhance the user experience in XR.

This enhancement could further facilitate user adoption of XR technology by improving comprehension and comfort of navigating an ever-evolving digital landscape.

The research aims to formulate an XR experience framework that is intuitive, biophilic, and responsive in design, tailored to the fluctuating and dynamic demands of users overwhelmed by technology. By portraying a contextually aware digital entity in a comprehensible form such as a fluffy cat, the user is assisted in bridging the knowledge gaps necessary for the adoption and utilisation of new technology.

This research builds upon the foundational inspiration presented in Ted Chiang's novella "The Lifecycle of Software Objects". Chiang's exploration of the relationship and ethical intricacies between humans and AI entities underscores the necessity for emotional assistance and guidance in a highly logical and structured digital world.

This investigation is further enriched by the work of Hodent, which underscores the importance of human cognitive processes and emotional responses when designing engaging digital user experiences. These insights are crucial in ensuring that Digital Entities are designed with a user-centric approach, thereby augmenting interaction and engagement within XR platforms.

Bogost's concept of procedural rhetoric guides the interactive and influential aspects of XR experiences. It contends that interactive experiences have the ability to present arguments and influence players, while offering the opportunity to challenge players' preconceptions and encourage them to contemplate topics in new ways. This is particularly poignant in a world where data and news are highly curated by algorithms designed to elicit views, whilst being rarely fact-checked and prone to misinformation.

A digital entity that acts as a counterbalance to the user's worldview posits an improved ability for a user to navigate a tumultuous digital environment. Sicart's emphasis on ethical game design ensures that the Digital Entities provide moral considerations and opportunities to explore the repercussions of decisions.

The research questions at the core of this project focus on the application of generative AI in enhancing user experiences in XR and the adaptation of XR environments using real-time data. The methodology is a blend of theoretical research, empirical prototyping, and iterative development. This project endeavours to contribute significantly to both academic and practical spheres.

Spanning a timeline of 3.5-years, the project will progress through the stages of theoretical research, prototyping, iterative development, comprehensive testing, and final evaluation. The importance of this research lies in addressing a key challenge of interactive development - the process of onboarding users for mass-market adoption.