Natural Menu Selection and Symbolic Input Using the MYO armband

A gestural symbolic input system for wearable devices

A user playing the game developed to test the text input system design


To come up with a novel approach for symbolic input for wearable technologies in scenarios where a physical keyboard cannot be used, and voice commands might be awkward. This design needs to be easily learn-able and transition the user from novice to expert quickly.

Working with the myo - Challenges

The MYO (by Thalmic Labs) seemed to be a good fit for input in such a scenario because it is minimally intrusive and can be used anywhere since it is worn on the hand and reads muscle activity, it is not limited by a confined physical workspace which is sensed.
But, it came with its own set of challenges.
There are very few simple poses that the MYO can recognize.
A lot of false negatives in gesture recognition, for a subset of the poses that are relatively easy to perform.
Need for resrved gestures due to the live mic problem.
When using the MYO, there is also no physical frame of reference.

Design Decisions

We combined the accelerometer and gyroscope data with the pose information to come up with a bigger vocabulary of gestures to support more interactions.
We designed the system to interpret one gesture for the same purpose at all times. For example, an open fist would mean one level up depending on the state of the system.

To deal with the problem of the lack of reference, we decided to keep the gestures relative to one another and not defined as absolute, leveraging Myo's ability to be sensed anywhwere.
We also designed for the sequence of gestures to always be the same so it can be memorized and easily performed .

click here for the blog related to this project.