Work Preparation:
We had our quarters presentation on Wednesday, successfully conveying the project goal and our three pipelines to the faculty. We also got a lot of feedback and reflected on them with the instructors at our weekly instructor meeting. Two things stood out during the quarters, one is that people are interested in seeing the failures with no masking, which means we need to show what doesn’t work as well as the ones that work. Another thing is that faculty stressed out the importance of documenting, as during the final we should not only show the visuals but also the development processes.
We also talked with the Petrol people and the AI game team, exchanging the information we have and discussing the potential tools that we could use.
After the quarters, we had a clearer vision of what to aim for the halves milestone.
Progress Report:
- Finished the team photo, poster, half sheet, and project description.
- Finished setting up the project website.
- Finished the quarters presentation and compiled the feedback we got.
- Met with the Petrol AI people and the AI Games team.
- Set our goal for the halves.
- Started greyboxing the mountain.
- Started generating the yin and yang cracking scene.
- Started working on the camera angles and the film sequence.
Research Results:
- Team Photo, Poster and Half Sheet with AI Help:
This week we used AI on the team photo, poster and the half sheet. We utilized the Midjourney art that ZhouZhou made as the base of the image. We experimented with Firefly’s “text effects” AI tool, but it was difficult to get it to output simplistic styles that were also legible. Like other AI tools, it always wanted to add more detail than necessary. This led to styles that did not fit what we wanted, and were too busy.
We also used photoshop’s AI generation to expand on the original image. It didn’t do a great job of matching the style. The colors were broadly correct, but the fidelity was not there. This is odd, especially since Photoshop’s AI generation works very well for actual photographs. It also wanted to constantly place the image in a frame, and we had no choice but to edit that out by hand.
The team photo utilized concept art for Pangu that was generated using Firefly and Stable Diffusion. Projected onto that image by Unboring were photos of our faces. Unboring did the best job among its competitors at transposing the faces, particularly when trying to transpose multiple faces at once. However, it tries to match the face being copied to be in the same pose as the face being replaced. This can lead to undesirable results given that we were not interested in the original expression.
- Model the Assets and Research More about AI Texture:
We AI modeled the mountains and applied AI textures (Poly and Texturelab), experimenting with the AI texturing tools and testing the results.
We also tested using Meshy to texture the manually made mountain assets.
- The results demonstrate that approach two is much more stable.
- Poly can produce some good textures, but then the textures are doing nothing with the model UV. It does show the right style and atmosphere.
- Meshy is super powerful and accurate, it can give us the right style and different styles in a short amount of time. It is very impressive.
- Meshy auto texture does have some problems, especially when the model has a lot of flat area. Plus, sometimes it produces more colors than you asked for.
- The model needs to have a nice UV and ambiguous area (such as flat areas) that confuses Meshy.
- Promising AI texture does not mean artists are not needed; AI can produce strong art styles, but they failed to meet some ambiguous needs, for example: in this scene, everything should be quite blurry, chaotic, and dim, we artists still need to do some tweaks, or just hand paint the models.
Plan for next week:
- Redraw the storyboard and fill in some scene transition details.
- Discuss with the instructors about our milestone goal.
- Continue working on the website and weekly blogs.
- Continue working on the camera angles and the film sequence.
- Continue working on greyboxing and hand modeling the assets.
- Start to model Pangu’s hands and rig them.
Challenge:
- Hard to estimate how long each approach will take and what potential problems we will meet throughout the semester.
- Hard to estimate the cost for AI tools, and how effective they will be.
- Need to think of better ways to document our research process.
- Need to show all three approaches and their differences on halves, need to convince the faculty that all three films can be completed by final.
- One member is sick due to Covid, and we need to update him on the project progress and feedback we got from quarters.
- Need to discuss our milestone goal with instructors.