This semester, the goal of Team ImmerX is to explore possibilities of blending the physical and virtual worlds, pushing the boundaries of storytelling and interactivity. This project also serves as the starting point for the future development of XR experiences. We started with research and development, playing MR games, prototyping interactions and playtesting, managing all of them into our interaction catalog. Then, we selected the top core mechanics that bring out the best experience, designing a demo experience.
Project Overview
ImmerX explores the future of storytelling by transforming real-world spaces into dynamic, interactive stages using mixed-reality technology; redefining how and where stories are told, pushing the boundaries of creativity, technology, and guest interaction. We aim to craft experiences where guests don’t just observe but actively shape the story, creating meaningful engagement.
Sprints before ½
Four Design goals:
Space, immersion, intuitive, wow moments
We have four design goals based on our prompt and sprints. We aim to fully use the physical space by extending the virtual world beyond the real one, focus on creating magical moments over a full game structure, make the experience immersive by blending virtual and real elements, and ensure intuitive interaction through both direct and indirect controls.
Three Aspects:
- Space, interaction, blend virtual and reality
Technical Art & VFX Overview:
Most of the technical art and VFX in our scene are designed to support interaction, both between characters and with objects. For character interactions specifically, we utilize VFX and shaders to provide reactive visual feedback.
Reflection and Learnings
To design a better XR experience, played and analyzed XR games, paying close attention to how they approached interaction and immersion. As a team bonding activity, we tried VR Squid Game. On top of that, we looked into past ETC AR, MR, and VR projects, and understand how they tackled similar challenges.
Through all these experiences, we learned a few key insights: First, even the simplest interactions—like grabbing objects can be surprisingly fun. Second, one of the most magical moments in XR is when the virtual world blends seamlessly into the real environment. Seeing virtual objects appear right in our actual space—like a spaceship landing in rooms—felt incredibly immersive and exciting. And finally, what makes an XR experience truly memorable are those wow moments—like being able to pull virtual items straight up from the floor. These interactions make the experience feel alive, like the virtual world is truly part of our own. We kept all these insights in mind as we moved into our explorations, experimenting with new ways to make interactions feel intuitive, immersive, and, most importantly, fun.
We utilized an R&D pipeline:
-We had 4 rounds of sprints fast prototyping 24 different interactions and effects with a demo project that implemented the selected interactions in a coherent experience, with frequent playtests and iteration along the way.
-We test our design ideas with generalizable playtesting data.
Summary
By combining guided setup, adaptive spawning, and scalable design, we created a flexible system that makes the XR experience functional, immersive, and user-friendly across a wide variety of home environments.
Conclusion: Key Insights – Lessons in XR Development
Throughout the journey, we’ve achieved many of the wonderful moments that players enjoyed. We also encountered many challenges, as well as aspects that we wish to continue developing.
- We checked out the headset the first day of the first week, and started a gold spike, testing out the giant creatures outside the window that worked pretty well, which carried along the entire semester.
- This is the time you spend to explore what could be possible and work well in the AR world before committing to an entire design that might not work really well.
- Things could be less intuitive in augmented reality because people see both their own space and objects which sometimes might be hard to notice the digital designs.
- Action early!
- AR is very different from other platforms, even different from VR.
- Test early before going too far into design. Don’t take for granted that things will just work.
- Limited technical capacity for high-poly meshes in virtual reality.
- Playtest frequently
- Don’t be afraid of giving players “too much” instructions. Enough instructions before they really carry on and have the freedom to explore.
- Things could be less intuitive in augmented reality than you expect.
- Certain gestures remain challenging to accurately recognize and implement effectively in virtual reality environments.
- Details are crucial for a good experience
- Blend the boundary between digital and physical world even more
- Consider interaction between virtual & virtual, virtual & physical
- Take advantage of the real world space, and the objects in your space
- Example: Decorating players own space, collection system
- Observe what fascinates players when designing “Magic Moment”
- Giant creatures
- Positive instant feedbacks for interactivities (sound, visual)
- Blend the boundary between digital and physical world even more
