This week, we aligned our ideas with both the team and instructors, and conducted our second shooting test, this time with more complete assets and a refined technical pipeline.

Compared to our previous test, we made significant progress. We worked within a more developed environment, integrating additional finished assets and arranging the layout based on our concept art. On the technical side, we upgraded the setup, using a newer version of the Vive Tracker and Unreal Engine, which provided a more stable connection and improved accessibility without relying on the CMU Local Area Network. This updated configuration also proved to be more flexible, allowing us to set it up in the cavern.

We conducted the shoot on Friday, but unfortunately, we faced a number of unexpected challenges. However, these setbacks provided valuable lessons, giving us the opportunity to reflect and improve our workflow for future shoots.

Shooting On Friday

We spent a significant amount of time in the green screen room, from 12 PM to 11 PM, navigating numerous unexpected challenges during our second shooting test.

Our biggest issue was the mysterious sphere that kept appearing in the scene and obstructing the view. Despite multiple attempts to delete it, the sphere persisted. We currently have two hypotheses for the cause: either Unreal Editor caches were the culprit, or the sphere was being generated during runtime, requiring a command line to remove it. We haven’t yet pinpointed the exact cause.

The second issue involved the layout mismatch between the nDisplay mode and Unreal Editor preview. In the Unreal Editor, our assets looked perfectly arranged, but when projected in nDisplay, where the camera is mapped to the real-world position, the scene looked significantly different. We spent most of Friday rearranging the assets to fix this discrepancy.

Additionally, we had to physically rearrange the green screen room to resolve position-matching issues. However, this introduced another problem—an unstable connection with the Vive Tracker. It’s possible that the close proximity of the TV and base station was interfering with the tracker’s stability. As a result, we had to repeatedly adjust the room layout to restore proper connections.

Lastly, we set up a DV camera to record the entire process, but when we attempted to transfer the footage to the PC, the files were missing. Despite being sure that the DV was recording, there were no video files on the device. We’ll need to troubleshoot this issue as well.

Despite spending the entire day trying to resolve these problems, there are still many issues left to fix. As a result, we’ve identified key lessons and tasks to focus on moving forward.

Lessons Learned and To-dos

• Debug the mysterious sphere!
  • This is a critical issue, but since it’s likely not hardware-related (like the Vive Tracker’s stability problem), we’re optimistic about resolving it. We plan to consult more experienced programmers and focus on Unreal VP templates as the potential cause.
• Develop a more complete plan before shooting (call sheet).
  • Friday’s production was a bit disorganized. We need to create a call sheet, like in regular productions, to prioritize tasks and stick to a clear schedule.
• Create a formal floor plan for the green screen room setup.
  • We need to formalize the setup and layout of the room, much like the cavern’s standardized template and setup.
• Arrange the environment using the Virtual Production (VP) perspective.
  • This is a crucial VP lesson—matching the virtual and real-world environments. To streamline this process, we might consider developing a small blueprint tool to act as a matching helper.

MetaHuman Character and Art Progress

Dress Creation & Rigging: Using Marvelous Designer, we replicated the character’s dress and successfully rigged it. This ensures the dress behaves correctly when integrated into Unreal Engine.

Hair Redesign: We also remade the character’s hairstyle to better match the model, refining the overall look.

Asset Refinement: Previous assets were adjusted to more accurately reflect their real-world counterparts, ensuring a more cohesive appearance.

Material Completion: Finally, we finished the material work, finalizing the look of our virtual character.

Camera Focus Syncing

We found a plugin to sync a Canon camera with unreal using LAN, but the model of the current camera was old and cannot support wireless connection. Steve said But we have set up a wired connection using the computer to control the camera.