During Week 05, our primary focus was refining the overall structure of the experience—both narratively and technically.
Gameplay & Narrative Development
We reorganized the entire gameplay flow and translated it into a clear storyboard format. This helped us identify pacing issues, interaction gaps, and moments where collaboration between players and the robot needed stronger motivation. Through this process, we made several adjustments to improve clarity, tension, and progression. We refined how information is revealed, how tasks escalate, and how cooperation becomes increasingly necessary as the experience unfolds.
This restructuring allowed us to align narrative beats with interaction mechanics, ensuring that story and gameplay support each other rather than operating separately.
Early Modeling & Scene Setup
We also began preliminary modeling and spatial setup. Initial scene blockouts were created to test scale, player positioning, and robot presence within the environment. These early models help us evaluate physical-virtual alignment and prepare for interaction prototyping in the next phase.
Technical Implementation
On the technical side, we completed the first working pipeline for controlling the Unitree G1 robot using Meta Quest hand tracking.
The system now includes:
- Hand tracking (30Hz, 25 joints per hand) on Meta Quest
- WebSocket transmission over WiFi to the host PC
- Inverse kinematics (Pinocchio + CasADi) to convert wrist pose into 7 joint angles per arm
- Finger retargeting (DexPilot) to map 25 hand joints into 7 robot finger joints
- Smoothing filter (Weighted Moving Average) to reduce jitter
- CycloneDDS communication (Ethernet) to send commands to the robot
- WebRTC visual feedback loop streaming the robot’s head camera back to VR
This marks a major milestone: we now have a stable real-time control pipeline from VR hand tracking to physical robot actuation.