The second week for Sprint 1. Since the painting prototype was almost finished last week, we ran the playtesting on Wednesday to collect feedback from users. For the garden prototype, we continue developing it and finished by this Friday. Also, on Friday we have a regular weekly meeting with Google.
Playtest
We ran the playtesting on Wednesday for the painting prototype at ETC. We divided our user group into two types. One is the Preferred Audience, they are the ETC Student who has prior knowledge in ML. The other type is General Audience, they are the ETC students who don’t have prior experience in ML. In total, we ran 16 playtests,8 for the general audience and 8 for the preferred audience. For each playtester, they have 10 mins to play with the painting prototype and 10 mins to fill out the questionnaire. We designed different questions for different groups.
Here is the link to the questionnaire. https://forms.gle/uRnuVahJL2TXWsYS9
Here are some sample questions:
- What did you learn about Machine Learning?
- What do you think the different K values are doing to the painting?
- Compared with a 2D set. Does the 3D representation explain the data better?
- What images worked well in this 3D? or what images looked better in a flat 2D format.
- Where do you think this particular model can be used?
- Would you be interested in seeing this in VR?
- What other platform do you want to see this project? (Phone/Website/AR/VR…)
After playtesting, we collected the feedback and analyzed them. Here is what we found.
Insights
Image Choice
Living creative images didn’t work well. People tend to choose images they want
Fun to play with
There was a “wow” moment when painting layers were distributed.
AR/VR Platform
Playtesters wanted to see this project move further in AR/VR platforms. (Possibly Gifs/Videos)
Feedback
Clear Guidance
Clearer UI showing step-by-step process.
Scale UI (Speed/Spread/Zoom) can be clearer.
Loading, Description
Without a loading bar, it’s confusing to know if the computer is processing.
The length of the description needs to be determined carefully.
Purpose
Where can this project move further?
Garden Prototype
For the garden prototype, we finished all the functions this week. Currently, the user can create new data points in the garden, check the details of each data point, choose the K value and the initial mean, and then see how the K-means algorithm runs step by step. We also added background music and animations to let the user feel more engaged. Here is the demo video: https://drive.google.com/file/d/1McygzJeNXk0o0ImsBl4mW_hAYsMYLuVZ/view?usp=sharing
We plan to run the playtest for the garden prototype next week. First, we want to see if showing the process of how an algorithm works can help users better understand the algorithm. Also, we want to see if flowers and gardens are a good visualization for the K-means algorithm. We also want to find the relationship between the text and visualization. For example, some questions could do good visualization raise the user’s interest to read the text. Finally, we want to compare two prototypes to discover the advantages and disadvantages of each prototype.
Meeting with Google
On Friday, we had a meeting with Erin, Sherol, and Matt from Google. We showed them our results from user testing and the progress of the garden prototype. They gave us a lot of useful information for our next step. Therefore, we decided to focus on making an engaging experience using ML for our sprint 2.
Here are their suggestions:
- Does not have to be limited to Machine Learning.
- The team can focus on data science/visualization
- The music prototype is interesting to explore
- Resources: CoCoNet, Magenta
- Use existing model created within Unity / available on the internet
- Language/Word related ML/AI is interesting to explore
- Example: Semantle. How can we find a better way to guess the word?