Work Preparation:
Next week is the Thanksgiving break. Since our core hours are on Wednesday, Thursday, and Friday, the break will leave us no time to work on the project next week. Thus, we added one more core hour day this week, so that we can well prepare ourselves for the Soft Opening Day. We updated both versions of the film, which are the versions that will be shown on Soft Opening Day, and we sent those to Panpan for sound edits and faculty consultants for review.
We finalized the character movement animations and worked on the the facial expressions and hair & beard movement animations. The full character (with hair!) and the new animations were implemented in the film with an updated angle now. We also altered the last shot so that the clouds do not look flat and the camera works great for the standing up animation now. Also, we updated the cloud dripping VFX and the dust VFX so that they will look better in the film. The AI rendering research goes well, we found some interesting colorful results, as well as one parameter that is giving us the exact black and white wood cut painting style we were looking for.
Progress Report:
- Updated the project website and weekly blogs.
- Added new research results into the documentation.
- Updated the documentation in the correct format.
- Met with Pan-Pan and updated her on the film we want to showcase on Soft Opening Day.
- Finalized the character animations as well as facial expressions.
- Adjusted some camera angles based on the character animations.
- Continued researching AI rendering.
- Updated the AI character texturing.
- Updated the mountain dust VFX.
Research Results:
- AI Rendering:
Following last week’s research, we should keep polishing the result by cleaning up the noises on each frame & improve consistency between frames. Since we are using normal, depth, and tile these 3 modules inside controlNet, and to keep consistent, we have to at least keep one of the modules between depth and normal. So the path is clear to polish the result, we try something different with the tile module.
The previous setting for tile module is shown in the screenshot. Last week, we have been trying to find a good parameter for control weight, now it is time to play with Variation.
This is what you get with Variation 8, so we start testing with different variation value.
Variation 6
Variation 3
Variation 22
Variation 32
After several experiments, variation with a value of 32 fits the best since the whole image is unified with less noises. The less the variation is the more tiling blocks there are within the image.
In addition to the noises, the over exposed area around the edge of the image needs fixing. So we need the second functionality of the tile module, resample.
We keep the normal and depth module unchanged, and used the default parameter for the tile module with the same picture we sent to it for color fixing. We use the generated picture under variation 32 as reference image for generation.
Resampled image
The result is amazing, the image after resampling is much cleaner as well as getting more details. I also tried to disable normal and depth module respectively to see which one of these are critical for keeping consistency.
Without Normal Module
Without Depth Module
Mind-blowing result, depth seems to be more important than normal, but both of these method have huge potential for style transformation.
But now we keep working with both normal and depth enabled for consistency. I also tried to use a different checkpoint model to see if the styles would change drastically. However, the result seems binary with two major styles generated, one is the one already have, and the other as follows.
Using CounterfeitV30 instead of darkSushiMix
This direction is also promising, we will explore it in the future.
- AI Texturing on character:
We used Meshy to do character full body texture this time.
Prompt: ANCIENT GOD STRONG, BLACK AND WHITE (oriental comic ink)
Prompt: ANCIENT GOD STRONG, BLACK AND WHITE (cartoon line art)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render (realistic hand-drawn)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render (realistic)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE (cartoon line art)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE (oriental comic ink)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE (realistic hand-drawn)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE(realistic)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE (2.5 D Hand-drawn)
Prompt: ANCIENT GOD STRONG, HDR, studio quality, realistic, ultra realistic, trending on Artstation, masterpiece, cinema 4d, unreal engine, octane render BLACK AND WHITE (oriental comic ink)
AI has trouble differentiating the front and the back of a character body/hands without hair, cloth, and hat etc. AI tends to texture lots of unwanted patterns/colors on the bare hands, but it can add accurate details on the knuckles of the fingers.
Plan for next week (after Thanksgiving Break) :
- Meet with Panpan for sound support.
- Compile the feedback we get from Soft Opening Day.
- Continue working on the website and weekly blogs.
- Do some last minute changes to the camera and greybox sequence.
- Do some last minute changes to the animation as well as VFXs.
- Continue test out different styles with AI rendering.
- Do more research on AI modeling and AI rigging.
- Prepare for the final presentation.
- Everyone go over the research documentation again.
- Post-mortem for the project.
Challenge:
- Hard to produce films of good quality with unpredictable AI bahaviors throughout the semester.
- Need to make sure we have some powerful shots in the film.
- Need to make sure we get useful feedback from the ETC Soft Opening.
- Need to keep Panpan on the same page for sound production.
- AI modeling is proved to fail, and need to shift focus to AI rigging for approach 2.
- A lot of changes in the character animations, which will also affect the camera sequence.