About Speakeasy
The goal of the Speakeasy project is to develop a fun, effective, and compelling method to learn new languages using the Minspeak symbolic icon system.
Minspeak is a dedicated hardware/software platform that allows people to communicate by representing words and concepts with sequences of icons. This was originally developed to help the communication-disabled speak more efficiently, and we are expanding its scope to include language learning on a tablet device.
Minspeak is a dedicated hardware/software platform that allows people to communicate by representing words and concepts with sequences of icons. This was originally developed to help the communication-disabled speak more efficiently, and we are expanding its scope to include language learning on a tablet device.
Our Client
SpeakEasy is sponsored by Semantic Compaction Systems Inc. developers of Minspeak, a language solution for people with speech impairing disabilities. They approached the ETC because they saw an opportunity to adapt their system to language learning.
My Role: Lead ProgrammerHow can we integrate the client's existing icon IP to create a Spanish language education game for English speakers? The client's icon IP is traditionally used in augumented speech tools for those with conditions like autism who have difficulty speaking. Icon sequences generate words in less button clicks than typing and relies heavily on motor mapping logic. My role is to satisfy all the requirement from the designers, developing app on tablet is very different from on the computer because of the memory limitation and the platform. We used jpct as our 3d engine which is able to load the animation of the model from maya. This is the most amazing part because most of the 3d game on android was not able to do this. |
|
Also One of the more unique challenges presented by a game where the player can form their own sentences is not only how easy it is for the player to mess up their sentence, but how many ways there are to do it! Everything from wrong-gendered pronouns to disagreeing pronouns and verbs to incorrect word order to valid sentences but wrong content…The way we handle this is with language parsing. We actually break down the player’s answer into its words, look at their parts of speech, and figure out what properties they have (gender, number, and person are the properties we care about in our prototype). We do the same for the known correct answer – we now have syntax token lists.
From there we launch into a series of structure comparisons, gates by which we determine what kind of wrong the player’s answer is. Depending on how their answer is syntactically incorrect or differs from the expected answer, we’re able to deliver a clear and targeted pre-authored response, which is customized based on the words and sentence in question. A diagram of the process (for the sample incorrect answer of “Ella la cocinan”, where “cocinan” does not agree with “Ella”): |