Digital Twin Integration & Immersion Polish

This week marked a major technical milestone for our project. We successfully mapped the updated character model onto the robot’s digital twin. Using the digital twin’s skeletal structure as the foundation, we adjusted and re-rigged the model to better align with the robot’s actual joint hierarchy and movement constraints. This significantly improved the consistency between the virtual avatar and the physical robot, reducing visual mismatch and enhancing embodiment. The alignment process required careful iteration to ensure that proportions, joint rotations, and deformation behaved naturally within the constraints of the robot’s real-world mechanics.

Beyond technical alignment, we focused heavily on immersion. We added background music and environmental sound effects to strengthen the atmosphere of the spaceship setting. Subtle mechanical hums, spatial audio cues, and ambient tones now support the emotional tone of each gameplay moment, making the experience feel more cohesive and alive.

Additionally, we introduced voice-based inner monologue cues that trigger at key moments. Instead of explicit instruction, these subtle narrative hints guide players when they appear stuck or uncertain. This approach allows us to maintain immersion while gently supporting player progression without breaking the experience.

Overall, Week 13 represented a shift from core system development to refinement and experiential polish. With the avatar mapping stabilized and audio layers integrated, the project is beginning to feel like a unified and intentional mixed-reality experience rather than separate technical components.

Add “SIUU”!!

ASAP Sim-to-Real Deployment

Repo: LeCAR-Lab/ASAP · Branch: sim2real/ · Env: asap_deploy (conda, Python 3.10) Goal: Get the G1 29-DoF locomotion + SIUU mimic policy running on the physical robot.

Sim2Sim first: Debugged MuJoCo viewer key sequence. Robot must be lowered with 8 first, then 9 to release the elastic band (at 3m height), then ] to activate the locomotion policy. Pressing i (init state) softens joints — must be avoided. Fixed DDS interface mismatch: config/g1_29dof_hist.yaml had INTERFACE: enp0s31f6 (real-robot NIC) instead of lo for sim2sim. Changed to lo — policy control began working. SIUU mimic motion confirmed: ] → ; → [.

Real robot deployment: Switched INTERFACE back to enp0s31f6 (workstation at 192.168.123.163, robot PC2 at 192.168.123.164). Hardware blocker resolved: The G1 EDU+ ships with a physical waist fastener (2× M5 screws + 2 fixing parts) that locks 2 of 3 waist joints → limits to 23 DoF, breaking the 29-DoF policy. Physically removed with Allen wrench. Disabled the waist motor lock in the Unitree Explore App (Settings → Robot), connecting phone to the robot’s own WiFi AP. Re-entered low-level mode: L2+R2 → L2+A → L2+B. Robot stood and responded to policy commands.

Policy files used: Loco: ./models/dec_loco/20250109_231507-…/model_6600.onnx Mimic: ./models/mimic/ Script: rl_policy/deepmimic_dec_loco_height.py Config: config/g1_29dof_hist.yaml

SIUU on the physical G1 confirmed working.