Sign Language Tracker
In response to the overfitting issue identified last week in our Sign Language Tracker, we implemented several strategies to improve model performance:
Model Adjustment Strategies:
We adjusted the model’s weights to prevent overfitting. This included incorporating dropout layers to regulate the learning rate, implementing early stopping to halt training before overfitting starts, and experimenting with the addition of regularizers to penalize complex models.
Data Augmentation:
To enhance the model’s generalization capabilities, we added noise data to our training set, including ‘thumbs up’, ‘thumbs down’, and various random gestures. This diversity in data helps the model learn to distinguish between intentional sign language gestures and non-sign movements.
We will keep testing and try to strike a balance between accuracy and generalizability, ensuring the tracker performs reliably across a wide range of inputs.
Two Sets of UI
We’ve introduced two UI setups tailored for left-handed and right-handed users. Players can now easily select their preferred hand via the settings on the main page, having a more personalized gaming experience.
Skeleton Experiments
This week, we conducted further experiments with the skeleton model, testing various colors, sizes, and levels of transparency. Of all the variations, a bluish color scheme was most harmonious with our scene. We also streamlined the design by removing superfluous dots from the skeleton, achieving a simpler and clearer visual.
UI Update
We’ve refined the UI for Mini Game 3, the game selection page, and the ready page, creating two versions for each to present to our client. After reviewing both options, we reached a consensus with our client in favor of the right version for each interface. These selected UI designs were successfully implemented into the game in time for our playtesting day as well.
For the ready page, we’re considering centralizing the hint image and the skeleton to eliminate the need for separate layouts for left-handed and right-handed users. This way, we would only need to swap the hint image to correspond with the user’s hand preference, whether it be for the left or right hand.
Hint Options & Parameters
We’ve refined the hint options across various levels and games, introducing the ability to customize the delay before a hint appears, particularly for the beginner level. This feature will be parameterized, allowing our client to adjust the timing according to future research needs.
| Beginner | Intermediate | Advanced | |
| Mini Game 1 | Hint appears after 2 sec | Hint appears after 5 sec | |
| Mini Game 2 | Hint appears after 1.5 sec | Hint appears only when the alphabet was not hit last time | No hint |
| Mini Game 3 | No hint |
Art Update
Intro Animation
We’ve decided to integrate comic-style animations before the start of each game to provide players with additional context on the narrative, aiming to enhance their engagement. Below is the introductory animation for mini game 1.
More 2D Assets
There’re also more 2D assets for the mini game 3.
Latest Demo
Playtesting Day
In preparation for our Playtesting Day, we created a survey with invaluable suggestion from our client, JT, and our advisors, Tom and Heather. They advised us on constructing a more comprehensible and effective survey. For instance, instead of soliciting binary responses (yes or no), they recommended asking participants to rate their agreement with specific statements. This approach allowed for greater variability in responses and could provide us with more detailed insights through follow-up questions. After several iterations based on their advice, our survey was significantly improved and proved effective on Playtesting Day.
The complete list of survey questions is provided here.
Our playtesters ranged across various age groups, from young children and teenagers to adults. The feedback we received was predominantly positive, highlighting the game’s fun and educational aspects. A significant portion of the suggestions focused on hand detection accuracy. These concerns were divided into two main issues: interference from other hands in the background and the model’s overly high confidence in its detections. We’ve organized all feedback into categories for more efficient resolution.
Furthermore, we’ve initiated testing of our data collection pipeline to better understand the data our client will be gathering. We will convert the JSON file into a CSV format, analyze and discuss the results more comprehensively next week.
