Week Overview
This week, we kicked off by gathering to discuss our plans and goals for the semester. Our primary objective is to explore various tools and technologies, develop proof-of-concept demos, and thoroughly document our process to contribute to the growing body of knowledge on virtual production.
To achieve this, we outlined a five-stage semester plan. Each stage will focus on testing different technologies and equipment, allowing us to gradually build toward our final outcome.Research and Exploration
Virtual/Physical World Matching in Final Production
Movement and Dynamic Content Matching
Miniature with Camera Synching/Physical Light Sourcing
Final to Demonstrate Lessons Learned
Deliverables by the end of the semester:
Comprehensive documentation detailing the entire Realtime Virtual Production Pipeline.
A festival-worthy demo that showcases the lessons learned, providing an immersive experience where guests can interact with and visualize your virtual production in real time.
Talk with SMEs
Talk with NEP
During the week we have connected with Zach from NEP. We consulted Zach about several technical questions of Virtual Production based on our previous research and established follow up contact. Zach answered questions on lighting environment and foot Lambert, providing us solutions and insight from a more practical and detailed perspective. He also answered our questions on camera tracking problems by giving us resources like Vicon, and Redspy.
Meanwhile, Zach also broadened our horizon on virtual production by suggesting we look at more examples and references in sports events, live shows, and even Vtubers.
Talk with Wlab
During the week we also reconnected with one of the supervisors Todd at a Virtual Protection Center Wlab about the opportunities of further academic and educational collaboration. Meanwhile, Todd went through with us all the technologies and aspects of virtual production that Wlab has and the support that they could offer us. We are still working on moving forward the future collaboration.
Some of the references may found here:
Online Research
General Study on Virtual Production
Generally Virtual Production(VP) is a filmmaking technique that combines the physical and digital worlds to create movies and television shows. It uses a variety of techniques, including LED or projection backdrop, motion capture, virtual cameras, in camera VFX etc. A regular virtual production pipeline could be briefly described as:
We also found some indie VP pieces using LED volume
And some reference using motion capture and virtual camera
Real Time-camera tracking and virtual camera
Green Screen cases
PRIDASK – Making of ‘Virtual Escape’ Music Video / Green Screen + Unreal Engine 5
Sci-Fi Short film | Behind the scenes | Unreal Engine 4.27 |
Case Study
We found a really interesting virtual production shot on miniatures that was published by the New Yorker called The Vandal(2021) by Eddie Alcazar, combining the stop motion animation and regular film production together.
The production crew called this “meta scope” of combining different media together of different scales in a single continued story. In its behind the scene series, the team showed us the process of establishing an LED screen behind the miniature environment to ast as the background and light source. The piece shot most of the established shot and medium shot in stop motion and closeup using real actors and actresses to create a sense of surrealism whereas in the story after the protagonist takes the pill.
This also gives an inspiration and creative solution of how to combine small scale virtual production on LED screen and traditional shooting, on set or using a green screen. After watching more product commercials productions like the Burger King advertisement, we came up with a potential direction and idea of making a combination of miniature and human actors in front of green screen of a short based on Alice in Wonderland where the character has been made into toys.