{"id":472,"date":"2026-03-16T20:07:16","date_gmt":"2026-03-16T20:07:16","guid":{"rendered":"https:\/\/projects.etc.cmu.edu\/soulos\/?page_id=472"},"modified":"2026-03-16T20:48:14","modified_gmt":"2026-03-16T20:48:14","slug":"week-05","status":"publish","type":"page","link":"https:\/\/projects.etc.cmu.edu\/soulos\/week-05\/","title":{"rendered":"Week 05"},"content":{"rendered":"\n<p><\/p>\n\n\n\n<div class=\"wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained\">\n<p style=\"font-style:normal;font-weight:500\">During Week 05, our primary focus was refining the overall structure of the experience\u2014both narratively and technically.<\/p>\n\n\n\n<p style=\"font-style:normal;font-weight:500\"><strong>Gameplay &amp; Narrative Development<\/strong><\/p>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">We reorganized the entire gameplay flow and translated it into a clear storyboard format. This helped us identify pacing issues, interaction gaps, and moments where collaboration between players and the robot needed stronger motivation. Through this process, we made several adjustments to improve clarity, tension, and progression. We refined how information is revealed, how tasks escalate, and how cooperation becomes increasingly necessary as the experience unfolds.<\/p>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">This restructuring allowed us to align narrative beats with interaction mechanics, ensuring that story and gameplay support each other rather than operating separately.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"1024\" src=\"https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/6d1236e1e3b0b162cb109f22dfd075c8-768x1024.jpg\" alt=\"\" class=\"wp-image-479\" style=\"box-shadow:6px 6px 9px rgba(0, 0, 0, 0.2);width:335px;height:auto\" srcset=\"https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/6d1236e1e3b0b162cb109f22dfd075c8-768x1024.jpg 768w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/6d1236e1e3b0b162cb109f22dfd075c8-225x300.jpg 225w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/6d1236e1e3b0b162cb109f22dfd075c8-1152x1536.jpg 1152w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/6d1236e1e3b0b162cb109f22dfd075c8.jpg 1279w\" sizes=\"auto, (max-width: 768px) 100vw, 768px\" \/><\/figure>\n\n\n\n<p><strong>Early Modeling &amp; Scene Setup<\/strong><\/p>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">We also began preliminary modeling and spatial setup. Initial scene blockouts were created to test scale, player positioning, and robot presence within the environment. These early models help us evaluate physical-virtual alignment and prepare for interaction prototyping in the next phase.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Technical Implementation<\/strong><\/h4>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">On the technical side, we completed the first working pipeline for controlling the Unitree G1 robot using Meta Quest hand tracking.<\/p>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">The system now includes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Hand tracking (30Hz, 25 joints per hand)<\/strong> on Meta Quest<\/li>\n\n\n\n<li><strong>WebSocket transmission over WiFi<\/strong> to the host PC<\/li>\n\n\n\n<li><strong>Inverse kinematics (Pinocchio + CasADi)<\/strong> to convert wrist pose into 7 joint angles per arm<\/li>\n\n\n\n<li><strong>Finger retargeting (DexPilot)<\/strong> to map 25 hand joints into 7 robot finger joints<\/li>\n\n\n\n<li><strong>Smoothing filter (Weighted Moving Average)<\/strong> to reduce jitter<\/li>\n\n\n\n<li><strong>CycloneDDS communication (Ethernet)<\/strong> to send commands to the robot<\/li>\n\n\n\n<li><strong>WebRTC visual feedback loop<\/strong> streaming the robot\u2019s head camera back to VR<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"1024\" src=\"https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-16-162406-800x1024.png\" alt=\"\" class=\"wp-image-480\" style=\"box-shadow:6px 6px 9px rgba(0, 0, 0, 0.2);aspect-ratio:0.7812552982265776;width:364px;height:auto\" srcset=\"https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-16-162406-800x1024.png 800w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-16-162406-234x300.png 234w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-16-162406-768x983.png 768w, https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-16-162406.png 912w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/figure>\n\n\n\n<p style=\"font-style:normal;font-weight:500\">This marks a major milestone: we now have a stable real-time control pipeline from VR hand tracking to physical robot actuation.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained\">\n<figure class=\"wp-block-video\"><video height=\"320\" style=\"aspect-ratio: 568 \/ 320;\" width=\"568\" autoplay controls loop muted src=\"https:\/\/projects.etc.cmu.edu\/soulos\/wp-content\/uploads\/2026\/03\/a3f45b63fb5bf48f584a5c04c4bf0cde-1.mp4\" playsinline><\/video><\/figure>\n<\/div>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained\" style=\"padding-top:var(--wp--preset--spacing--50);padding-bottom:var(--wp--preset--spacing--50)\">\n<p class=\"has-text-align-right\"><a href=\"https:\/\/projects.etc.cmu.edu\/soulos\/week06\/\">Week 06\u2192<\/a><\/p>\n<\/div>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>During Week 05, our primary focus was refining the overall structure of the experience\u2014both narratively and technically. Gameplay &amp; Narrative Development We reorganized the entire gameplay flow and translated it into a clear storyboard format. This helped us identify pacing issues, interaction gaps, and moments where collaboration between players and the robot needed stronger motivation. [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-472","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/pages\/472","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/comments?post=472"}],"version-history":[{"count":9,"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/pages\/472\/revisions"}],"predecessor-version":[{"id":495,"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/pages\/472\/revisions\/495"}],"wp:attachment":[{"href":"https:\/\/projects.etc.cmu.edu\/soulos\/wp-json\/wp\/v2\/media?parent=472"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}