Thank you for following our journey to create “Rhyme Rider”! This week, we presented to the faculty our final product. Check out the presentation:

As we’re finishing the semester, here’s so reflection on our process and progress:

What went well:

We believe that we succeeded in building a fun level within this rap game concept that uses Gen AI in clever ways. The product shows how Gen AI can be used in a game and proves that there should be more rap-based games out there. 

Our programmer and narrative designer worked closely to create understandable and efficient documentation to prompt the AI. That relationship is why we were able to have the AI work in multiple parts of the game. Having a narrative technical designer who understood the game design and had programming skills was essential. 

Our artist had worked together before, and it showed in their pipeline. 2D designs were made into 3D models within the week. They were in constant communication about the relationship between the 2D and 3D components, which created a nice union of both art assets. 

A lot of our success can be attributed to our teamwork. We spent a third of the semester just visualizing what the game would be and working through the tech challenges. Once the larger game decisions were made, it was easier to divide and conquer. 

Some of the team wanted to have “no meeting days” to focus on their individual tasks, while others wanted everyone to be available for collaboration during core hours. We had some real heart-to-hearts about our preferred working schedules and respected our time/responsibilities outside of the project. 

The trust that we had with each other also created a learning environment for people to work beyond their role. We wore multiple hats when needed, like our sound designer/animator, because we were invested in the project and had the support of our fellow teammates.

What could have been better:

We had some issues importing towards the end of the semester. Our 3D models had a lot of off-center pivots, which made marking colliders very tedious work. Since our characters were designed with baggy clothes, we had to add subtle deformations when animating. That meant having to re-texture the model and adding more complexity to our animation importing pipeline. We were importing .abc animations instead of .fbx. This then made us have to use the animation timeline feature in Unity rather than the regular animator controller. 

AI was very fun to work with, but very hard to perfect. In every aspect of our AI usage, there is an edge case we aren’t accounting for because AI is too free and unpredictable. For example, in our AI score feature, you could essentially rhyme “a” with itself 4 times and still get a decent clarity score, because although “a” doesn’t make sense at the end of a sentence it is still a word in the English language and the AI feels like that’s enough. Scoring is a whole issue in itself because we’re helping players write some of the rap, so the AI is scoring the AI. That’s like asking to grade your own homework. We also really wanted to add more rap concepts like internal rhymes or slant rhymes, but AI simply doesn’t understand how to create or detect them. 

Lessons Learned:

There was a lot to learn since no one on the team had built a game that uses Generative AI or a game that involves music. We had to do a lot of research on what AI was capable of and what elements of rap/rap culture we wanted to emulate.

Technical Lessons:

There’s a lot Gen AI can do, and a lot it’s just not good at yet. You have to constantly remind it of small things like “ Don’t respond more than 3 sentences”. AI is only as smart as we tell it to be, and it is very literal. For instance, we wanted one of the audience preferences to be that they like ocean metaphors. We prompted the AI, “ You are the audience and you like ocean metaphors”. The audience characters start using ocean metaphors instead of saying they liked the ocean metaphor.s. We later changed the prompt to be “ You like when rappers use ocean metaphors.”.  It’s good a mimicking, but there needs to be enough content to mimic. It’s not so great at justifying its answers, and sometimes, simply hallucinations rules. 

Word docs were easy for the AI to read, but the formatting that you used was important. If there was a table in the doc with examples, you had to say, “read this table below and use the rows as rap examples”. We looked through a lot of different documentation about how to prompt AI. We used both system and user prompts, but it was a dance between the two. 

We had an intricate system flow of the AI components. None of them worked in isolation because we were creating a persona. This meant that we had to deal a lot with remembering previous interactions. It made the character more (for lack of a better term) “alive”. 

Design Lessons:

We were told very early in our development that “AI doesn’t replace good game design”. It was important to make sure our game flow was interesting enough before we thought about which parts we could use AI. If we wanted to just use AI, this would be a research and discovery project, not a game design one. We would only need programmers and not designers or artists. 

We spent the first couple of weeks trying to figure out what the core gameplay would be, and then we paper prototyped it. With those parts solidified, we were able to make small tech demos and start using AI.

Our sound design was extremely important because of the setting of the game, a rap battle. Our sound designer had to be diligent about her research into G-funk and was able to connect the sound to gameplay. For example, the idle animation of the character nods with the beats of the music. Or, the music during the rap battle is in count with a 4-bar verse. 

Art Lessons:

We were often told by the professors that our art style was too “young” for our audience. We learned through playtesting that older players didn’t feel hindered by the art style; rather, they thought it added a lot to the “coolness” of the game. We figured out that if the art is cohesive enough that we can take more creative liberties with the style. 

Conclusion:

We loved this project and we want to keep working on it. There are a lot of features we would like to add to the game, such as our next character, XO Soul (a jazz rapper),  a performance cutscene during the rap battle, a shader, more writing techniques, etc. We are very proud of the work we were able to achieve. The lessons we learned were very valuable, especially with the gaming industry looking more and more toward AI.