Week 7 [Sprints to 1/2!] – February 28th, 2025

Week 7 [1/2!] – February 28th, 2025

It’s the week mixed with both prototyping the project and preparing for the presentation of ½. Team ImmerX continues finishing up the sprints which were designed for the first half of the semester, aiming to serve as a process of understanding the key moments for our project focus, finding the core mechanics to work on for the rest of the weeks. At the same time, artists started on building assets and special effects for the demo project, which is designed to be produced during the second half of the semester. This demo project will include key selected core mechanics, designed with themes for players to explore and interact with.

Trailer

This semester, our goal is to explore possibilities of blending the physical and virtual worlds, pushing the boundaries of storytelling and interactivity. This project also serves as the starting point for the future development of XR experiences.

We adapt our production pipeline with half research and development, following with a demo project for the other half of the semester. We started with playing MR games, prototyping interactions and playtesting, managing all of them into our interaction catalog. Then, we selected the top core mechanics that bring out the best experience, designing a demo project with a designated theme. 

To design a better XR experience, played and analyzed XR games, Starship Home , First Encounters, paying close attention to how they approached interaction and immersion. As a team bonding activity, we tried VR Squid Game. On top of that, we looked into past ETC AR, MR, and VR projects, understand how they tackled similar challenges. Through all these experiences, we learned a few key things: First, even the simplest interactions—like grabbing objects can be surprisingly fun. Second, one of the most magical moments in XR is when the virtual world blends seamlessly into real environment. Seeing virtual objects appear right in our actual space—like a spaceship landing in rooms—felt incredibly immersive and exciting. And finally, what makes an XR experience truly memorable are those wow moments—like being able to pull virtual items straight up from the floor. These interactions make the experience feel alive, like the virtual world is truly part of our own. We kept all these insights in mind as we moved into our explorations, experimenting with new ways to make interactions feel intuitive, immersive, and, most importantly, fun.

Based on case study and quest 3’s capability We come up with 4 sprints including 24 interactions and effects.

Sprints!

• Automatic Registering
• Building Up Environment
• Magic Lens 
• Gesture Recognition

The first sprint is automatic registering. We use quest 3 to register Wall ground ceiling window and poster. We use bullpen as a starting space. The yellow part indicated by the diagram is replaced with a virtual forest environment, where we are able to see through with depth from our physical room.

Sprint 1 – Evolving on Window

Upon that, we advance our exploration with window. Since it takes a crucial part in our design. We focus on how to make interactions happening between the inside and outside world so that they can blend together. We designed a flow of actions:

Sprint 2 – Building Up Environment

For the second sprint, we shift our focus from virtual environment to our physical setting as indicated by the yellow part of the diagram, which is the furniture, floor and wall in our room. And the main goal is to build up from our physical environment. So here are two main effects: the shadow illusion traveled through the wall and floor using decal projection. The growing vines upon the furniture that can be procedurally generated by a tool.

Sprint 3 – Special Rendering & Visual Tricks

For the next sprint, we created several unique visual tricks for players.
Instead of directly changing objects in the scene, we played with what the player sees in the virtual space. For example, we added a magic lens that reveals hidden elements floating in the air and a virtual flashlight that lights up both the real and virtual worlds, making exploration more engaging and immersive.

Sprint 4 – Gesture Recognition

Beyond those limitations, we also conquered some challenges. In order to better understand how mixed reality works, we divided the scene into four parts which are inside objects, passthrough, outside objects and skybox. After testing them one by one, we figured out their overlay relationship. This helps us realize some interactions we want to achieve.

First of all, we decreased the render queue of the virtual window material below the passthrough mesh in order to see the outside objects and skybox behind it. As you can see, the passthrough is blocking the skybox, however, if we place a virtual window which has a lower render queue before it, the camera will render the skybox first. This is how we make our virtual window effect.

The other one is the magic lens effect. We want players to reveal some hidden objects through the lens. To realize this, we used a stencil mask. The mechanic of a stencil mask is that if we set the stencil ID of our lens and the hidden object layer to the same value, we can see the whole hidden object layer through the lens. In this project the stencil ID is 1. We were supposed to use the same way to make the crack which lets the player view another world and set the stencil id to 2. However, it somehow doesn’t work properly. Since we only need the skybox of the other world, we attach the skybox to the crack’s shader. In this way, the crack becomes a common object with a skybox as its shader which can be revealed directly by the camera without using any masks.

Over four sprints, we gained valuable insights from playtesting and meeting with ETC professors. We learned from four key perspectives:

1. Player Persona: We observed how players behave in the headset and gathered their thoughts and motivations.
2. Space Testing: We tested the demo in different types of rooms (offices, Hunt Library, apartments) to understand the potential and limitations of AR in various room setups.
3. Sprint Evaluation: We reviewed all 24 sprint pieces based on feedback and technical feasibility, deciding whether to proceed or block ideas.
4. AR Design Insights: We identified key factors for a successful AR experience, such as the “WOW moment” and “freedom of exploration.”

These insights have guided us when designing the final project.

Our core mechanic is a travel machine that takes the player’s entire home on an adventure to various worlds. Players preview these worlds through expandable “cracks”, choose a destination, and launch the machine. Upon arrival, they will see the new world through their actual window and experience unique stories in each one.

Players will also have two fun portable tools to assist them in exploring the new worlds:

• A Collector that can be placed anywhere at home to collect and bring in different items from the outer world.
• A Vision Gate that can be attached to wall/ceiling/ or ground to see through it. 

We believe these 3 core mechanics could serve as a strong foundation for developing fun interactions in the future.

Above is a top-down map of our basic room setup: we designed the experience for two connected rooms with at least one window.

After placing the travel machine console, the player can look through their room windows, walk around in space, and use the collector and vision gate to further explore.

Up to this point, we already started building the demo project for the second half of the semester. We designed 5 different themes with visual references and our own concept arts. From the technical side, our programmer is implementing the core mechanics from the previous sprints, utilizing the prototypes to the maximum, and our artists are in the production process of the art assets. By the end of the semester, we will have a complete experience that demonstrates the magical moments.  

Week 8 ToDo

Art

• World 1: Dark Pink Theme
• Core mechanic: Collector, Travel Machine Console

Tech

• Core mechanic: Collector basic functions 

Design

• Core mechanics functions
• UI functions
• Main interactions in World 1, 2-in progress

Production: Playtest to Refine Workshop