This week was the anticipated CAVERN Jam, where our toolkit will meet its first set of users! In the two-day event, there were 8 participants bringing 6 different worlds, each exploring different ways of interactions and immersion methods in this space. While technical difficulties did emerge, overall feedback was clear: Spelunx made developing for CAVERN dramatically easier, and people are excited to make more experiences here.
Six Unique Worlds
Kicking off on Monday and showcasing on Tuesday, within a tight 24-hour timeframe, 8 participant (including both our team members, other ETC students, and even faculty and staff) created 6 worlds, and drew over 20 attendees in the showcase. Here we’ll breakdown what we learned from each experience.
Alex Hall Flesh Wall
Alex, a multimedia artist in our cohort created an unsettling experience inside of a wall made out of flesh, where giant eyeballs will stare at the person wearing the tracker, creating an eerie sense of being watched constantly. Along with loud and scary surround sounds screaming into the space, the experience was very scary and very immersive, and all people gasped in excitement and fear when the experience first started. (And this is verified in every future demo we’ve done to guests that came to ETC)
Two things we’ve learned:
- Tracking works for everyone else other than the person wearing the tracker. While this is a known design constraint that stemmed from our rendering code, when a person wears the tracker on their head, from their perspectives, the eyeballs will not stare directly at them, but at a space above and behind them. However, for every other people in the space, they will perceive the person as being watched correctly. This opens up discussion for multiplayer experience design.
- Distortion of the scene is minimal when art assets were placed extremely close to the screen. When looking at our sample scene, we knew that as people walk around the space, there will be some degree of distortion of the scene due to our rendering solution. However, this did not happen in Alex’s Flesh Wall, and that was because of how close the art assets was placed near the screen.
Alex is an artist, and the only code needed for this to work was a LookAt built-in method for the eyes to follow the Vive Trackers. She spent a total of 10 hours, with most of the time used recording sound effects and voice lines, to create the experience. In fact, the CAVERN setup took only 20 minutes! This is a huge achievement for our toolkit.
Josh’s Frogs Choir
One of Spelunx team members, Josh, created a musical interaction where four Vive Trackers are used to trigger singing frogs, and the volume and pitch will change based on player’s proximity to the screen. The design naturally encouraged different play styles: players could place a tracker in one frog’s zone and leave it singing, or multiple people could move around dynamically to shift the composition. A dead zone area is carefully spaced at the center, so it is also a possible interaction when one person wanted to stay there and trigger the frogs like a conductor.
This piece demonstrated CAVERN’s multiplayer potential that utilizes physical space, which is a major difference between the CAVERN and other virtual reality systems. With all the sound and art assets, this experience was created in under 4 hours.
Jing’s Bubble Game
One of the programmer students at ETC, Jing, created a bubble interaction game where players could use Vive Trackers to repel and pop bubbles that flies towards you. While simple, the interaction was surprisingly engaging as people found joy physically reaching out to the bubbles, and even tried to compete against each other on who popped the most bubbles. In addition, because the bubble was transparent, the players were able to see both the real world as well as the virtual environment. It blended the physical and digital seamlessly.
As a programmer, Jing incorporated our sample scene as the base environment art assets, and focused on the coding side of development!
Terri’s Head-Tracked Anime Girl
Terri, Spelunx’s own rendering and graphics programmer, created a technical proof-of-concept demo of head tracking. The scene featured a dancing anime girl that the person wearing the head tracked Vive tracker could see from different angles. This was a huge achievement, as it was a novel rendering solution that enabled simple implementation without needing to move the whole world to get the same effect (what previous teams used, which severely affects performance). This head tracking gold spike demonstrated that CAVERN is capable of enabling more complex interactions.
Winnie’s Little Match Girl Immersive Storytelling
Inspired by a previous mixed-reality project from Visual Story course, Winnie’s world utilized surround audio along with visual storytelling to create an immersive space. The experience began with the players lighting up a candle using a Vive Tracker, and a voice circulating the space in an eerie tone. After the candle is lit and the voice lines were played through, the scene will transition into a dreamlike world with a giant whale flying around, and the Vive Trackers turn into a bubble.
Though technical issues prevented some elements from functioning (the Vive Trackers were set too low so candles couldn’t be lit, and the computer was set to stereo instead of surround sound) the core environmental design remained effective. Even without full interactivity, attendees found themselves immersed in the scale and atmosphere of the scene, demonstrating that CAVERN’s visual and beautiful music alone gives powerful potential.
Grace & Selena’s Femto Bolt Tracking
As part of Anamnesis team, another CAVERN project team focused on live-action interactive filmmaking using the Orbbec Femto Bolts, Grace and Selena experimented with Orbbec instead of Vive Trackers for motion tracking. However, because our toolkit currently only supports Vive Trackers, they faced integration challenges. Still, this project shed lights on what difficulties we might face when we moved to integrating Femto Bolts.
Mike & Bryan’s Space Game
Our consultant Mike Christel and ETC Senior Research Programmer Bryan Maher collaborated on a project featuring colorful balls shooting to you the same way a fireball might fly towards you during a space fight. While the project was halted due to busy schedules, they gave us invaluable feedback on their initial development process and documentation suggestions, such as tutorials on binding a Vive Tracker, and removing the manual Tarball installation.
Reflections and Refinements
CAVERN Jam concluded with overwhelming praises, not just from the attendees but also those who came to the showcase. Everyone was inspired to create more experiences on CAVERN, and this is exactly our goal.
Throughout the jam, our team was able to quickly push bug updates within a short hour because of our choice to host the package through UPM instead of GitHub.
On the development side, the feedback we received show that confidence levels for developing on CAVERN went up from 3 out of 5 to 4 out of 5, and participants quoted the documentation as well as the CAVERN previewer as a highlight that accelerated their development and testing.
On the other hand, there are areas identified for improvements:
- Manual tarball installation was cumbersome. We did not have enough time to put that into the package before the jam, but now we have automatic installation configured.
- Error messages that came from SteamVR settings not present on developer’s computer should be suppressed in order not to confuse them, since the errors will disappear immediately when they are on CAVERN computer, and the game should run normally on their computers.
- We should allow more time for participants to test in the CAVERN, so errors such as incorrect audio configuration will not be present in the showcase.
More updates
Sound configuration 5.1 to 7.1
After fixing the computer configuration itself, we also realized that 5.1 setting does not align as well with the actual physical speaker setup. Instead, having 7.1 while disabling the two surround speakers and the center speaker is more like our intended quadraphonic setup.
Sample scene refinements
To refine the sample scene before Halves, we:
- Added a butterfly animation to better demonstrate movement across the curved screen and into the space.
- Changed terrain the skybox assets from gray box materials to the ones we made.
- Tested shaders by adding a water shader to the river in the scene.
Next Steps
CAVERN Jam validated the strengths of Spelunx, while also highlighting key areas for refinement. With this, we are ready to present our half semester achievements to community and faculty in next week’s halves presentation. Stay tuned for more updates.