Get in touch: etc-chatbot-ai@lists.andrew.cmu.edu



Week 6 – 2nd Playtest

Summary

Hey all! This week has been an exciting week for us! We conducted our first digital playtest in Hunt Library on the main campus, and polished our build for the upcoming halves presentation.

Game Design

Our first digital playtest went really well and it gave us a lot of insights. We had 5 groups of playtester(s) interacting with our vertical slice character, Adam, where they asked various questions, used interrogation strategies to draw a conclusion of if Adam was guilty / if there was another suspect they’d find guilty and wanted to interrogate. Below are our notes:

  • Adam did not deal well with non-sequiturs or follow the flow of conversation sometimes, might be formatting/prompt related
  • Players in general resonated well with the format and asking questions, found it fun
  • Many did not use tone wheel, felt like they were already getting information, or just forgot, tone wheel did affect how questions were asked in a good way when used, probably more a UI/UX and tutorial issue but needs further investigation
  • The onboarding is lacking for shorter playtests, need a more concise explanation to get players into the game quickly that they can refer back to as well as explain mechanics and goals
  • Most identified Adam as angry, bitter, irritated, except the play testers that broke him with non-sequiturs. They seemed to understand the relationship with Hope and Adam well
  • Adam is way too verbose, he probably needs much shorter/less dense answers both as it feels like too much for players to take in and seems out of character
  • Most noticed Adam’s changing tone, though it did not seem to impact what information he was giving much, and all states being equally verbose made it less noticeable
  • Repeated phrasing and some other issues means we probably need to do hyper parameter tuning
  • In general playtesters seemed interested to talk to the other characters that Adam talked about
  • 3 out of 5 said that answers were mostly as expected, 1 said it was mostly as expected with one or two notable exceptions, 1 playtester group basically fully broke it
  • Players really want example questions to ask
  • Got quite a few bugs of mixing up characters or ascribing relationships to them, probably due to lack of examples
  • Two person playtests are very effective for getting people to verbalize and change up strategies
  • The playgroup that broke it kind of got back on track, but it was pretty broken at that point, giving some validity to the idea that players will push the limits and if they find it works buy in

Programming

This week we implemented a new placeholder feature: Hardcoded Keyword Detection. this feature allows our character to respond (by increasing stress value) to certain topics, in addition to the input tone. It is currently implemented using a Keyword Search on player’s input and matching it with specific keywords hardcoded by the team, but it’s planned to be eventually replaced by a AI-driven method.

Below is a feature breakdown of our AI Characters.

Build

Over the week we also made a few changes on our User Interface and character context, which we think could be best presented with the latest build video we recorded.