Because the project will include multi-room experiences. We believe crossing doors is one of the necessary experiences. To make the door opening action smooth and believable, we are going to combine the features of OpenCV, ARUCO Tags, and Phidgets, and discover how to make these different technologies work together. This research will provide strong support in the future when we install the experiences. Thus, the Door Occlusion Demo is not only a utility research, but also the first step for the team to get through our pipeline.
OpenCV is the go-to computer vision library. Our current plan is to use it for ARUCO Tags, which are physical markers placed around our environment to help us recalibrate position and orientation, combating drift. We have managed to get position and orientation information from the tags, and are now working on positioning an environment based on that information.
For the door Occlusion demo, we are using special shaders that tell the Quest whether to render real-world data or virtual-world data. This way, we can model the position of the hallway and door in our Unity environment, apply this shader, and the Quest will show the user the virtual world only where we want it to
Since we are building an experience in a space that we can control, using external sensors and trackers is something we are exploring. We are using Phidgets to build a prototype with a rotation sensor to track the angle of a door hinge. With this, we can align a virtual door with the physical door, allowing for virtual objects to be occluded by a moving real-world object. The challenge with using Phidgets and the Meta Quest is the wired connectivity. To work around this, we have connected the Phidgets to a Raspberry Pi and are using a network server.