Design Challenge
Last week, we faced the challenge of balancing an engaging game experience with the research needs specified by our client. On one hand, we aim for a game that keeps players engaged and facilitates sign language learning through enjoyable experiences. On the other hand, our client prefers to minimize variation in individual experiences for more controlled data collection and analysis.
We’ve developed two strategies to accommodate both gameplay and research objectives:
A Toggle Between Normal and Research Modes
In agreement with our client, we’ve decided to keep the research and gameplay aspects separate, connecting them with a unique user code.
This way, for research participation, our client can direct users to a Google Form survey. Completing this survey grants users access to the game and a user code. By default, our game operates in normal mode, but there will be an optional input box for users to enter a code if they have one. When they start the game and input this code, the game switches to research mode, automatically transmitting their gameplay data to our backend server for research analysis. Another benefit of this approach is we can track data for each user, regardless of the number of times they access the game, as each player is assigned a specific code.
Additionally, our client has clarified how the data should be stored. We will initially save the CSV file on our localhost and then upload it to his Google Firebase server.
Parametric System
To address the distinct needs of research data collection and engaging gameplay, we’ve identified the necessity of different variable settings. For instance, in our second mini-game, the track speed was designed to fluctuate within a range, slowing down when players miss a letter to reduce frustration. However, from a research standpoint, a variable speed curve dependent on player performance and lacking consistency does not facilitate result gathering. So our client preferred a continually accelerating track without any slowdowns.
To resolve such discrepancies, we’ve decided to implement a parametric system within our game. This system allows us to extract specific parameters (e.g., hold time, track speed increase rate, timer, elements on & off, etc.) for control. We could create a specific set of parameters tailored to research purposes, which our client can also adjust for future studies. Meanwhile, we could have another set of parameters with a gameplay focus, aimed at encouraging players to persist in playing and learning the ASL alphabet.
Our client has agreed with this strategy and suggested a list of variables to control, including:
- How many letters/words people play with (all games)
- Gesture hold time to move onto next trial (for all games).
- Whether hint is provided or not provided (for all games)
- Initial speed for game 2.
- Amount of speed increase for correct trials (game 2)
- Amount of speed decrease for incorrect trials (game 2)
- Whether to show the hand landmarks on the screen (all games).
- When to show the hand landmarks on the screen (all games).
- If data is saved or not saved (all games).
We’ll finalize more details during next week’s meeting.
Model and Hint Image Iteration
This Monday, our team conducted a workshop where each member recorded our gesture signs for every letter, creating new training data. We captured screenshots of our hands, converting each into a single line of landmark data. In total, we amassed over 140,000 new data points, adhering to strict standards for each letter. This will help establish a solid ground truth for our model, allowing us to incorporate more tolerance cases after playtesting.
With our model newly trained on the latest data, we’ve also updated the hint images to ensure they align with the model’s results. We opted to present all signs from a consistent front angle, showing the perspective of a player looking into a mirror. For letters with critical details visible from other angles, such as “G” and “H,” we’ve included smaller images to highlight these specifics, as demonstrated in the right-hand version examples.
We will create two sets of hint images, one for the right hand and another for the left hand, essentially mirroring each other. To ensure accuracy in shape and direction, we took screenshots of ourselves performing the signs and then traced the outlines in Adobe Illustrator.
Mini Game 2 Update
Environment Layout
In response to last week’s feedback on the environment in mini game 2, we explored three approaches to align the setting more closely with our narrative.
Firstly, we adjusted the proportions between the cat and the buildings to enhance the miniature effect, moving away from a realistic city appearance.
Secondly, we experimented with procedural modeling in Houdini to create the ambiance of a Lego block city.
Lastly, we introduced real-world objects like toys and bottles into the miniature environment to disrupt the realistic vibe. The proportions shown in the following image will also be refined.
Upon review, we believe the third approach may offer the best outcome and is the easiest to implement. We plan to update the skybox and introduce an intro animation that sets the scene with the cat dreaming about racing through the miniature world.
Track Speed
As mentioned in the first section, we will implement two different sets of parameters for the track speed change rate, each set corresponding to either a research focus or a gameplay focus.
Demo
Mini Game 3 Update
Word List
We presented a list of words to our client and advisors for review. Based on the feedback received, here are some recommendations:
- Remove words that contain repeated nearby letters, such as “apple.”
- Eliminate words that could cause confusion or have alternative names.
UI Update
Result Page
Game Selection Page
Meeting with John
This week, we had a meeting with one of our consultants, John, and received the following feedback:
- The UI colors need to be clearer to distinguish between selected and unselected states.
- Clarify hint images further – consider adding an arm, body, or using a live photo.
- The skeleton style should be revised.
- Incorporate audio feedback.
- Adjust the hold time buffer – make it a parameter that can be manipulated.
- Enhance the letter selection page with more detailed sign illustrations.
Skeleton Update
We’ve updated the skeleton style, focusing on its color and line weight. Next week, we will adjust its position, scale, and central point for further improvements.
