![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/FdnLidhWYAUkzCX.webp\")
One of the key findings at this stage is that sign language typically involves dynamic movements of both hands to express phrases or words, often combined with facial expressions. Considering our project scope and schedule, we plan to focus more on static poses and single-hand signs for the first half of the semester. Once we’ve achieved basic goals, we can evaluate the feasibility of incorporating the phrase-based movements and see how much further we can develop in this aspect. Our client JT also agreed with our plan.<\/p>\n\n\n\n
In terms of phrase-based gestures, we might use checkpoints for the movement sequence detection, which would potentially impact the accuracy of the detection system. The tradeoff between system accuracy <\/strong>and vocabulary richness<\/strong> is also something we need to think about later.<\/p>\n\n\n\n Additionally, we found several language learning methods and we tried to incorporate some ideas like shadowing learning into our design as well. <\/p>\n\n\n We reached out to Prof. Golan Levin at the main campus and he gave us some suggestion as follows:<\/p>\n\n\n\n We also made connections with the head of the ASL Department at Western Pennsylvania School for the Deaf thanks to Anthony. Although we haven’t schedule a meeting yet, we’ve compiled a question list for sign language professionals.<\/p>\n\n\n\n <\/p>\n\n\n\n Sign Language General<\/strong><\/p>\n\n\n\n Accuracy<\/strong><\/p>\n\n\n\n Contents\/Topics<\/strong><\/p>\n\n\n\n Teaching<\/strong><\/p>\n\n\n\n Learning<\/strong><\/p>\n\n\n\n <\/p>\n\n\n\n We came up with four different pitch ideas this week before our client meeting.<\/p>\n\n\n\n <\/p>\n\n\n <\/p>\n\n\n <\/p>\n\n\n <\/p>\n\n\n After discussions with JT, we summarized a few key points:<\/p>\n\n\n\n Based on the feedback we got, we will work on some rapid prototyping this following week.<\/p>\n\n\n\n Unity – Python Communication<\/strong><\/p>\n\n\n\n The two-way communication between Unity and Python has been established and our programmers are working on the API part to make it easier to use across multiple programmers.<\/p>\n\n\n\n Gesture Recognition Demo<\/strong><\/p>\n\n\n\n We’ve trained our own ML (Machine Learning) model of gesture recognition. It started with four simple different gestures and we then collected data of 24 alphabets by inputting around 6000 images. The accuracy of the model is not high enough yet, but it gives a general idea and shows the promising future as long as we collect more data from various sources and improve the model.<\/p>\n\n\n Sign Language Research Since we didn’t know much about sign language, we conducted research on sign language mainly including the three parts: One of the key findings at this stage is that sign language typically involves dynamic movements of both hands to express phrases or words, often combined with facial expressions. Considering our project scope […]<\/p>\n","protected":false},"author":2,"featured_media":58,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/posts\/40"}],"collection":[{"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/comments?post=40"}],"version-history":[{"count":13,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/posts\/40\/revisions"}],"predecessor-version":[{"id":234,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/posts\/40\/revisions\/234"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/media\/58"}],"wp:attachment":[{"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/media?parent=40"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/categories?post=40"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/projects.etc.cmu.edu\/questure\/wp-json\/wp\/v2\/tags?post=40"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/learningtheory.webp\")
Connect with Consultants <\/h2>\n\n\n\n
\n
Sign Language Question List<\/h2>\n\n\n\n
\n
\n
\n
\n
\n
\n
Pitch Ideas<\/h2>\n\n\n\n
![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Questure-Pitch-1.0-1.png\")
<\/figure><\/div>\n\n\n![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Questure-Pitch-1.0-2.png\")
<\/figure><\/div>\n\n\n![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Questure-Pitch-1.0-3.png\")
<\/figure><\/div>\n\n\n![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Questure-Pitch-1.0-4.png\")
<\/figure><\/div>\n\n\n\n
Tech Update<\/h2>\n\n\n\n
![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Hand_Gesture_Recognition_1_24_2024_2_37_44_PM-edited.webp\")
![\"\"](\"https:\/\/projects.etc.cmu.edu\/questure\/wp-content\/uploads\/2024\/01\/Hand_Gesture_Recognition_1_24_2024_2_37_27_PM-edited.webp\")