{"id":205,"date":"2023-11-10T18:49:42","date_gmt":"2023-11-10T18:49:42","guid":{"rendered":"https:\/\/projects.etc.cmu.edu\/ailluminate\/?p=205"},"modified":"2023-11-13T18:56:36","modified_gmt":"2023-11-13T18:56:36","slug":"week-10-november-10th-2023","status":"publish","type":"post","link":"https:\/\/projects.etc.cmu.edu\/ailluminate\/week-10-november-10th-2023\/","title":{"rendered":"Week 10 &#8211; November 10th, 2023"},"content":{"rendered":"\n<p class=\"has-small-plus-font-size\"><strong>Work Preparation:<\/strong><\/p>\n\n\n\n<p>This week, we compiled the feedback we got from playtesters on the Playtest Day, and the whole team including Pan-pan reviewed the feedback together. Gladly, without any prior information on the film story, just by looking at the film one time, people are able to understand 80% of the film. Even though they do not understand Pangu god, they viewed him as somewhat of a giant. With the detailed character model in later, we are confident that the story will be conveyed even better.<\/p>\n\n\n\n<p>For the visuals, we agreed that the AI rendered film is a little bit too de-saturated, and thus the contrast between colors is small. The gray and white color palette makes it hard to read and visually-tiring for the audience. Also, the majority of the playtesters enjoy the opening scene in the AI-rendered film, the colorful universe scene. Thus, we are planning to add more colors into the later scenes in the AI rendered film, hopefully it will make the film more visually appealing. We have no idea how it would impact the noise control, and thus we will experiment with it more the next few weeks.<\/p>\n\n\n\n<p>For the AI textured film, people love the details in that film, including the texturing details as well as the VFXs. However, they also pointed out that the clouds and mountains kind of blend together in the chaos scene, as thus we need to alter the mountain texture a little bit so that there will be some differences between them.&nbsp;<\/p>\n\n\n\n<p>People also mentioned that some background music needs to be included as soon as possible. There are also some changes that can be made to give the current SFXs a more powerful sense.<\/p>\n\n\n\n<p class=\"has-small-plus-font-size\"><strong>Progress Report:<\/strong><\/p>\n\n\n\n<ul>\n<li>Updated the project website and weekly blogs.<\/li>\n\n\n\n<li>Added new research results into the documentation.<\/li>\n\n\n\n<li>Met with Pan-Pan and updated her on the sound feedback we received from Playtest Day.<\/li>\n\n\n\n<li>Updated the character animations.<\/li>\n\n\n\n<li>Adjusted some camera angles based on the character animations.<\/li>\n\n\n\n<li>Updated the Yin and Yang scene as well the hands scene.<\/li>\n\n\n\n<li>Continued researching AI rendering.<\/li>\n\n\n\n<li>Updated the AI character texturing.<\/li>\n\n\n\n<li>Did a trailing VFX and implemented it into the film.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-small-plus-font-size\"><strong>Research Results:<\/strong><\/p>\n\n\n\n<ul>\n<li class=\"has-small-plus-font-size\"><strong>AI rendering:&nbsp;<\/strong><\/li>\n<\/ul>\n\n\n\n<p>Last week we had Canny out, it\u2019s time to have something new in. Controlling lighting is an important topic that couldn\u2019t be ignored when making animation, so do AI-generated ones.&nbsp;<\/p>\n\n\n\n<p>First, we still begin with the same two units we have been using, normal and depth. But this time we need to change our way of thinking, instead of putting our reference image as the one waiting for repainting, we directly feed the reference image into the ControlNet unit.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image.png\" alt=\"\" class=\"wp-image-207\" style=\"width:838px;height:304px\" width=\"838\" height=\"304\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image.png 1600w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-300x109.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1024x372.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-768x279.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1536x558.png 1536w\" sizes=\"(max-width: 838px) 100vw, 838px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4.png\" alt=\"\" class=\"wp-image-213\" style=\"width:990px;height:557px\" width=\"990\" height=\"557\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4.png 1600w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4-300x169.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4-1024x576.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4-768x432.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-4-1536x864.png 1536w\" sizes=\"(max-width: 990px) 100vw, 990px\" \/><\/figure>\n\n\n\n<p>This will do the same job as previously they worked, transferring these reference images into their normal\/depth information. The image fed to the repainting part now changed, this time we are feeding a lighting reference into it.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11.png\" alt=\"\" class=\"wp-image-221\" style=\"width:800px;height:535px\" width=\"800\" height=\"535\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11.png 1600w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11-300x201.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11-1024x685.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11-768x514.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-11-1536x1027.png 1536w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1.jpeg\" alt=\"\" class=\"wp-image-209\" style=\"width:654px;height:436px\" width=\"654\" height=\"436\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1.jpeg 480w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1-300x200.jpeg 300w\" sizes=\"(max-width: 654px) 100vw, 654px\" \/><\/figure>\n\n\n\n<p>We use this photograph of mountain tops as a lighting reference to tell AI which part should be lit or not.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1.png\" alt=\"\" class=\"wp-image-210\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-1-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">AI-generated image based on lighting reference<\/p>\n\n\n\n<p>The generation seems to have more details and also we can clearly see the lighting condition has been reflected in the image, however, the composition has been greatly affected by the lighting reference image. We need to increase the control weight of our controlNet unit to have more control over the composition.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2.png\" alt=\"\" class=\"wp-image-211\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Weighted AI-generated image<\/p>\n\n\n\n<p>With a fairly good setting on depth and normal unit, it\u2019s time to add some new spice to the controlNet. This time we are using the Tile unit, and we are feeding a black-and-white shanshui painting as a reference for the unit.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3.png\" alt=\"\" class=\"wp-image-212\" style=\"width:744px;height:367px\" width=\"744\" height=\"367\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3.png 1600w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-300x148.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-1024x506.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-768x379.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-1536x758.png 1536w\" sizes=\"(max-width: 744px) 100vw, 744px\" \/><\/figure>\n\n\n\n<p>Tile unit has two main functionalities, resampling and color fixing. Even though we need color fix at this time, we first test with how resampling works.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-5.png\" alt=\"\" class=\"wp-image-214\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-5.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-5-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-5-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Resampled by Tile unit<\/p>\n\n\n\n<p>The resampling well reduces the noises in the image and adds details to it at the same time. It could be considered as a method for denoising and upscaling during post-processing.<\/p>\n\n\n\n<p>Our main focus this time is on color fixing with tile units.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-14.png\" alt=\"\" class=\"wp-image-225\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-14.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-14-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-14-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Color fixed by Tile Unit<\/p>\n\n\n\n<p>The result is surprising, the rim light and the high-contrast background perfectly reflect the lighting condition. But compared to the well-reflected lighting, the overall color palette is not close enough to the reference image we fed to the Tile unit. So we decided to adjust the control weight to have better results.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-7.png\" alt=\"\" class=\"wp-image-216\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-7.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-7-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-7-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Control Weight 1.2<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-9.png\" alt=\"\" class=\"wp-image-219\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-9.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-9-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-9-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Control Weight 1<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-10.png\" alt=\"\" class=\"wp-image-220\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-10.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-10-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-10-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Control Weight 0.4<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-6.png\" alt=\"\" class=\"wp-image-215\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-6.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-6-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-6-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Control Weight 0.6<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-12.png\" alt=\"\" class=\"wp-image-222\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-12.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-12-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-12-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Control Weight 0.7<\/p>\n\n\n\n<p>Around 0.7 is the best parameter after several experiments. And now since we have the correct parameter for color fixing, we need to use the correct lighting reference as well.<\/p>\n\n\n\n<p>Take frame 505 as sample, we replaced the lighting reference with the one we lit inside UE5.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13.png\" alt=\"\" class=\"wp-image-223\" style=\"width:870px;height:489px\" width=\"870\" height=\"489\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13.png 1600w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13-300x169.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13-1024x576.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13-768x432.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-13-1536x864.png 1536w\" sizes=\"(max-width: 870px) 100vw, 870px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">UE5 lighting reference<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"960\" height=\"536\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-15.png\" alt=\"\" class=\"wp-image-224\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-15.png 960w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-15-300x168.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-15-768x429.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Color-fixed and lighting-corrected AI-generated image<\/p>\n\n\n\n<p>The result is quite promising as it could even generate fog and clouds based on the lighting reference.<\/p>\n\n\n\n<p>Next, we take part of the sequence to test out the overall performance, and each gray-box image has a corresponding lighted reference to have a better lighted result.<\/p>\n\n\n\n<p class=\"has-text-align-center\"><img decoding=\"async\" loading=\"lazy\" width=\"600\" height=\"338\" src=\"https:\/\/lh7-us.googleusercontent.com\/ez4L2TvRYOLS8v7wU3fIX9xHF465Y7jTgaay9fFMC2Hpj0obDrP4LGoChwqPA-TNnQL5T1QRGQb34GY0mGpxRn9V3Zs9KdoUSimHCZ7nZe4R2svWtQW5QX98EFkxAGW4qic3DZFCJUfc1Ad985oqkYI\"><\/p>\n\n\n\n<p class=\"has-text-align-center\">Color-fixed and lighting-corrected AI-generated animation<\/p>\n\n\n\n<p>It\u2019s obvious that the animation has a lighting atmosphere according to the lighting reference, however, since we are playing at 30fps, the noise is too strong that couldn\u2019t be ignored. We need denoising.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" loading=\"lazy\" width=\"600\" height=\"338\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image.gif\" alt=\"\" class=\"wp-image-228\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Denoised and color corrected by Davinci Resolve<\/p>\n\n\n\n<p>Using the denoising method we previously mentioned, we get a rather good animation compared to its original one. The next step will be applying this lighting technique to the whole animation.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/lh7-us.googleusercontent.com\/iRpMhe46ALCuxbVTfXdcgkPLghBhwbI9DtD92AJkv8dmPuhWboN12ISEIT-EuxFpIzxRVGCn79U3sAL6e3RT6mJ_TyXR9Z136DY5b-YR9DRcVuX786K4IIoq_QaWW2pTY5R90YGA-Z5496xISb6-8-s\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Animation with noises<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/lh7-us.googleusercontent.com\/caeN5xMjesjMNAoei6l6WKrMIFsCRfIua_ez3wWPBAFqDcUDltxWgPqpCM9lhagDFzXcuLbzjTFLTs9K_NaKN-kFDdPbjmTZQutcZsKv73oiHhoPRNTEP7q9hGmJVpX3aR65kyfM_1hSK3EsLVWuVJY\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Animation denoised using Davinci Resolve<\/p>\n\n\n\n<p>Based on the built AI rendering pipeline, we tried to add more details to polish the animation as far as we can. One of the most important factors is lighting, and it is always something hard to achieve consistency using AI. Reversing the way we have been generating images, we are able to use light reference to tell AI how the lighting is, and by using the Tile module inside ControlNet, we are able to do color fixing to make the coloring more consistent. The next step is to further polish the current result.<\/p>\n\n\n\n<ul>\n<li class=\"has-small-plus-font-size\"><strong>AI texturing:&nbsp;<\/strong><\/li>\n<\/ul>\n\n\n\n<p>We used Meshy to AI texture the hand and full body models we got.<\/p>\n\n\n\n<p>Prompt: a hand &#8211;style ancient, 4k, hdr, high quality, hand texture<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image.jpeg\" alt=\"\" class=\"wp-image-208\" style=\"width:430px;height:521px\" width=\"430\" height=\"521\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image.jpeg 1258w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-248x300.jpeg 248w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-846x1024.jpeg 846w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-768x930.jpeg 768w\" sizes=\"(max-width: 430px) 100vw, 430px\" \/><\/figure>\n\n\n\n<p>Prompt: a hand &#8211;style chinese ink painting, black and white, 4k, hdr, high quality, hand texture<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-8.png\" alt=\"\" class=\"wp-image-217\" style=\"width:427px;height:553px\" width=\"427\" height=\"553\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-8.png 1143w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-8-232x300.png 232w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-8-791x1024.png 791w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-8-768x994.png 768w\" sizes=\"(max-width: 427px) 100vw, 427px\" \/><\/figure>\n\n\n\n<p>Prompt: a hand &#8211;style ancient, 4k, hdr, high quality, hand texture<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2.jpeg\" alt=\"\" class=\"wp-image-218\" style=\"width:431px;height:563px\" width=\"431\" height=\"563\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2.jpeg 1187w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-230x300.jpeg 230w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-784x1024.jpeg 784w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-768x1003.jpeg 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-2-1176x1536.jpeg 1176w\" sizes=\"(max-width: 431px) 100vw, 431px\" \/><\/figure>\n\n\n\n<p>Prompt: a hand &#8211;style ancient, 4k, hdr, high quality, hand texture<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3.jpeg\" alt=\"\" class=\"wp-image-227\" style=\"width:432px;height:570px\" width=\"432\" height=\"570\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3.jpeg 1124w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-227x300.jpeg 227w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-776x1024.jpeg 776w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-3-768x1013.jpeg 768w\" sizes=\"(max-width: 432px) 100vw, 432px\" \/><\/figure>\n\n\n\n<p>Prompt: BLACK AND WHITE HAIR ANCIENT, WATER PAINTING STYLE<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter is-resized\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16.png\" alt=\"\" class=\"wp-image-226\" style=\"width:450px;height:431px\" width=\"450\" height=\"431\" srcset=\"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16.png 1593w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16-300x288.png 300w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16-1024x982.png 1024w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16-768x736.png 768w, https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-content\/uploads\/2023\/11\/image-16-1536x1472.png 1536w\" sizes=\"(max-width: 450px) 100vw, 450px\" \/><\/figure>\n\n\n\n<p class=\"has-small-plus-font-size\"><strong>Plan for next week :<\/strong><\/p>\n\n\n\n<ul>\n<li>Meet with Panpan for sound support.<\/li>\n\n\n\n<li>Continue working on the website and weekly blogs.<\/li>\n\n\n\n<li>Finalize Pangu\u2019s movement animations.<\/li>\n\n\n\n<li>Work on Pangu\u2019s facial expressions as well as hair and beard movements.<\/li>\n\n\n\n<li>Research more on AI rendering and iterate on rendering the film.<\/li>\n\n\n\n<li>Improve the last scene camera angle and the background.<\/li>\n\n\n\n<li>Improve the Yin &amp; Yang scene and use different layers to improve the crack movements.<\/li>\n\n\n\n<li>Add the trailing VFX into the film.<\/li>\n\n\n\n<li>Provide an update on both versions of the films and give to Pan-Pan on time.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-small-plus-font-size\"><strong>Challenge:<\/strong><\/p>\n\n\n\n<ul>\n<li>Hard to estimate how long each approach will take and what potential problems we will meet throughout the semester.<\/li>\n\n\n\n<li>Hard to estimate the cost for AI tools, and how effective they will be.<\/li>\n\n\n\n<li>Need to think of better ways to document our research process.<\/li>\n\n\n\n<li>Need to make sure we have some powerful shots in the film.<\/li>\n\n\n\n<li>Need to prepare for the ETC Soft Opening.<\/li>\n\n\n\n<li>Need to keep Panpan on the same page for sound production.<\/li>\n\n\n\n<li>AI modeling is proved to fail, and need to shift focus to AI rigging for approach 2.<\/li>\n\n\n\n<li>A lot of changes in the character animations, which will also affect the camera sequence.&nbsp;<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This week, we reflected on the feedback we received during Playtest Day, and we made some visual changes as well as sound alterations.<\/p>\n","protected":false},"author":1,"featured_media":206,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"_links":{"self":[{"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/posts\/205"}],"collection":[{"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/comments?post=205"}],"version-history":[{"count":1,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/posts\/205\/revisions"}],"predecessor-version":[{"id":229,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/posts\/205\/revisions\/229"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/media\/206"}],"wp:attachment":[{"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/media?parent=205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/categories?post=205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/ailluminate\/wp-json\/wp\/v2\/tags?post=205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}