Week 8 Post-Halves – Mar 17th, 2023

Coming back from Spring Break, our team began focusing on what we needed to address based on the feedback we received from Halves.
We kicked off the week by meeting with our instructors and discussing what remaining aspects of our experience we needed to focus on. 

Now that our halves goal of making levels 2 and 3 were met, our team could focus our entire energy on building the tutorial level. The main goal of this week was to finalize the design for this level and how to most effectively convey the dance moves to a user. Using our preliminary design as a foundation and following a guided video on the back exercise provided by our clients, we iterated our key ideas for teaching and came up with the following:

  • Teaching the dance through segmented sections to prevent overwhelming the user and allowing them to digest each key component of the exercise
  • Adding visual cues through animated arrows and lines at key poses to demonstrate motion, tension, and release
  • Using verbal guidance to help guide a user through each pose like in an actual dance class
  • Highlighting the motions for contraction and release to follow the core principles of the Paul Taylor Dance Company

 

Then, based on these core ideas, we created a presentation drafting each of the core moments we picked out from the guided video

Sample slides detailing design concepts

From the programming side, one of the major issues we focused on tackling was the accuracy of the model animation as generated by the pose estimation model. Unfortunately, the ML model had many limitations in interpreting the poses from the back exercise video, specifically the more extreme motions that were “unnatural” for a human to perform normally.

In this week, the first focus was to reduce the noise in the motion generated by the model. Previously, this was being done through Unity via a simple Kalman Filter, which would take into account the positions of previous points in order to “smooth out” the later points. However, using a plain filter had the unfortunate side effect of delaying movements and reducing their intensity, as peaks and fast motions were mellowed out due to being averaged/affected by previous points.

To bypass this, we used a different algorithm for removing noise that would be more effective at preserving the motion intensity based on the paper ‘A Denoising Method of Human Pose Estimation Based on Reconstruction from Keyframes’ (2022). Here, we swap from being a position-based algorithm to one that is velocity and acceleration based; then, using the mean acceleration throughout the entire video, we filter out points with unreasonable accelerations (usually sudden spikes due to incorrect motion capture). The AcRDP method presented in this paper also preserves motion extremes, reducing the issue noted before with dampened movements.

Example of cleaned data points outputted by the algorithm from the x-axis

Along with these two main tasks, the rest of the team continued with iterating and developing the framework for the remainder of the experience. Programming focused on creating the framework for our new tutorial level, and art continued developing our 3D model and UI updates.

For the next week, the entire team expected to be out of town to attend the GDC conference. We took this as an opportunity to request some of the final assets we needed from the client, including additional Back Exercise recordings from different angles and more specific requirements along with BGM soundtrack to use for the existing levels.