We started out this week with Halves! It was very exciting to show off our work so far this semester and get to see all the amazing work that other teams are doing. It also made us realize how much more work we hope to get done the rest of the semester, especially before Playtest Day at the end of week 10. So, we made a schedule for weeks 8-10, which we hope will get us on track to finish a few experiences and integrate them into a rough-draft full experience. We have broken up the work so that we can polish the painting experience, create a bare-bones experience where the guest can control time (manipulating the hands of a physical clock to change time of day and grow a plant), and add a simple interaction in the hallway (a character in a virtual room who will move toward you whenever you look away), all in weeks 8 and 9. Then, we will have week 10 to polish and integrate before Playtest Day on March 28. While this is a lot to do, we have most of the foundations already in place and have outlined a clear path forward.
However, this week, we focused on finishing the first version of the painting experience. This involved fixing a bug with the ArUco markers; moving the marker in weird ways produced unexpected results in pose estimation (i.e. moving it away from the camera in the Z axis caused the pose to move up in the Y axis as well). We eventually found an issue with the camera intrinsic matrix Meta gives us, which meant we needed to change a setting in the camera API. After doing a few more tests to ensure we did not miss anything else, we finally confirmed that ArUco marker tracking is bug free!
With this, we were able to finish a rudimentary version of the painting experience. We created a virtual wall which matches the Jam o Drum room wall, and aligned it plus a virtual painting to the physical painting frame we have. We integrated the black and white filter we had last week, which we activate when the guest sticks their head in the virtual painting. We also switched the image the painting displayed between two premade options when the guest’s head leaves the painting, completing our first draft of the experience. In the future, we want to integrate a camera which will dynamically update the painting to exactly match what the guest does with the physical painting (as opposed to switching between two premade options), making the experience feel more real. We did the basic setup, getting webcam images to be wirelessly sent to our computer, but did not complete Unity integration before the end of the week.
We also started on some of the art aspects for our “control time” experience. Here, we want the guest to move the hands of a clock, and have time seemingly move back and forth accordingly. To create this effect, we want the lighting to change (as if the sun is moving) and a plant to grow. We created the start of a plant growing animation, and started to look into volumetric lighting in Unity and VR. We found a library with VR support for this lighting, and integrated it into our project. It looks pretty good so far (see below), and we are confident we can implement it into a final experience in a satisfactory way!