Coming back from spring break, we immediately hit the ground running! On day 1, we discovered a new SIGGRAPH submission category that fit our project so perfectly that we decided to still try to submit it before the 24 hour deadline. At the same time, it was the time to process feedback from Halves presentation and refine our long-term plans beyond simply toolkit delivery. In addition, most of our team members would attend GDC the next week, and we would need to prepare for showcase on the ETC booth. Finally, with South Fayette’s March 25th Showcase approaching, we also had to ensure our software as well as their hardware are well-tested and ready for presentation.
SIGGRAPH Spatial Storytelling
On Monday afternoon, we stumbled upon a new SIGGRSPH session called Spatial Storytelling. It celebrates realtime XR technology and emphasizes showcasing not just the cool and fun graphics but also the process of enabling or developing that technology. We immediately realized that this was the best place for our toolkit to shine.
However, there was a problem: it was due the next day at 6pm. Luckily, with halves just past, we had enough content to rapidly compile into a submission, including the very detailed mathematical derivation of the camera that Terri wrote before halves. We also decided to create a video even though it was optional, as we knew that without it, CAVERN capabilities cannot be shown best.
In the 24-hour sprint (less if counted sleep and elective times), some of us did video shooting, video scripts, and some of us did extended summary, abstract, and overview. our collective strengths came together brilliantly to clearly communicate the impact and importance of both the CAVERN as a novel XR space and our innovative solution to enable it more with our toolkit. We were also exceedingly fortunate to have our consultant Mike Christel, as well as advisor Drew Davidson to proofread our submission so many times. We finally submitted everything before the deadline, and we’re extremely proud to have it at the best quality we could have created in such a short timeframe!
The following showcases everything we submitted:
Synopsis
The CAVERN Toolkit is a collection of Unity 6 software tools and supporting documents that enables students to create immersive XR storytelling experiences for a low-cost multiplayer CAVE. With improved rendering capabilities and user-friendly tools, the toolkit creates a comprehensive foundational pipeline that minimizes technical overhead and expedites onboarding.
Abstract
The CAVERN Toolkit is a collection of software tools, Unity packages, and supporting documents that enables students to create immersive XR experiences even with little to no prior XR background. The CAVERN is a low-cost multiplayer CAVE with a 270° curved screen, 3D stereoscopic projection and a 20-foot diameter play area located at Carnegie Mellon University. These unique qualities of the CAVERN allow for multiple guests to physically explore an XR world together.
Previously, the platform was underutilized since the development pipeline was confusing, had poor rendering performance, and posed a steep learning curve for student developers. Grounded in user needs, the CAVERN Toolkit creates a comprehensive foundational pipeline that is modular and extendable. With this new pipeline, the onboarding time for the CAVERN development was reduced from several weeks to under 30 minutes.
The toolkit includes a new rendering solution that supports head-tracked stereoscopic rendering in all directions. A system was made to abstract away the previously tedious process for incorporating motion-tracking devices for interaction input and configuring spatial audio for immersive sound design. To increase accessibility for students and beginner developers, a simplified interface was implemented within the Unity 6 editor, enabling setup with a single click. Additionally, several sample projects were created to serve as inspirations and jumping off points. All of the above are bundled into modular packages hosted in the Unity Package Manager (UPM) and accompanied by extensive documentation and tutorials.
The CAVERN Toolkit significantly reduces the technical overhead of the CAVERN and allows developers to focus on what matters most: exploring creative spatial storytelling that blur the digital and the physical, giving everyone from high schoolers to experienced professionals the opportunity to make polished XR experiences that transport audiences into new realms.
Video Submission
Extended Summary
Mathematical Derivation of the camera
Reflecting on Halves feedback and planning for the future
Our halves feedback was extremely positive, with around half of the faculty stating that we exceeded expectations, as we were the first ever ETC team to have a playtesting session as large as a game jam arranged before halves. In fact, instead of simply asking what our next step for the toolkit development within this semester is going to be, faculty are asking questions about “How does the toolkit remain useful after we leave, and how can other users outside of ETC utilize it?”. This is a huge milestone for us, as it showed that our project was not a one-time thing, but recognized as living, growing, and extendable beyond just us, and even beyond just ETC!
Our purpose was clear: make Spelunx toolkit sustainable while testing extensively with as many different kinds of users as possible. In addition to fully supporting South Fayette, we also decided to reach out to main campus students who are interested in CAVERN, and have another game jam, with our next step features enabled.
Back to Toolkit development
By the end of the week, we shifted focus to back toolkit updates. We added the interaction building blocks scripts where users can easily one-click add a Vive Tracker interaction with the simple building block scripts we created. This currently includes: Follow motion (GameObject directly follows Vive Tracker), CAVERN mirror (a GameObject mirrors Vive Tracker as if CAVERN screen is a mirror), Evasive motion (a GameObject shies away from the Vive Tracker). This interaction building block kis in addition to our currently one-click Vive Tracker setup for users to quickly test and prototype simple interactions. The choices of interactions came from hearing the requests of experiences in the CAVERN, as well as insights from previous developers, who we conducted interviews with earlier in the semester.
In addition, to support South Fayette during Digital Promise conference on March 25, the first toolkit refinement was to support different CAVERN sizes, as South Fayette’s CAVERN is larger than ETC’s.
Looking ahead
With SIGGRAPH behind us and faculty confident in our direction, we’re now entering a new phase of development. We have more features to add, more documentations, more user testing, and all will be showcased and demonstrated in various demoing opportunities and cavern jam 2.