TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning (Lieberman & Breazeal)

Summary:

This work is motivated by the idea of motor learning being improved by feedback over all joints, which is not available from a human instructor, especially in a large class setting. Their system includes optical tracking (based on reflecting markers on their suit), tactile actuators (vibrotactile feedback -- by pulsing a series of actuators around a joint, an order for rotation is communicated), feedback software (triggers feedback when error between the learned joint angles and current angles is high enough), and hardware for output control. Their user study compared people learning an action with only video instruction to learning it with both video and tactile feedback from the suit. They found that subjects' error in performing the motions was reduced by a statistically very significant amount (numbers around 20%).

Discussion:

I like the idea of using tactile feedback to help learn actions that involve the whole body, like dancing, but I wonder if there would be a problem with not emphasizing the important joints more than the unimportant ones. It seems to me that in dance, some amount of stylistic variation is acceptable (though I haven't spoken to any professional dancers about how important it is to have every joint bent at exactly the correct angle). Or a short person engaged in ballroom dancing with a tall person would have to position at least the arms differently than a pair of dancers of the same height. It might be possible to offset this by training the system with several different experts, and it might be slightly less of an issue with a solo activity with no props.

I also think it would be very interesting to see, as they ask when discussing future work, whether human attention can take in information from more joints of the body all at once and react effectively to it.

Wiizards: 3D Gesture Recognition for Game Play Input (Kratz, Smith & Lee)

This paper describes a game where players make gestures to cast spells against opponents with a dueling wizards theme. Different gestures can be combined in series to make different sorts of spells. They use the Wii controller and feed its accelerometer data into an HMM gesture recognition package. They had 7 users perform each gesture over 40 times, and they found that having 10 states for a gesture in the HMM gave over 90% accuracy. After 20 training gestures from one user, recognition is over 95% (or 80% for 10 gestures). Recognition without user-dependent training data was found to be around 50%. Evaluating gestures in their system works at near real-time for 250 gestures, but training HMMs takes more time.

Discussion:
This sounds like it would be an interesting game to try, though I don't know how well you could convince all players to input a lot of training data so as to get remotely palatable recognition rates. It could work for a single-player game, if the design of the training session was sufficiently clever and fun, but as the focus is on being a multiplayer game, you don't want to make players who have played before wait for an hour while the new player goes through tons of sample gestures. It might be worth trying to have important features of a given gesture that can be communicated to the user, e.g. the most important features of this spiral gesture are that it is circular, on a plane, and never crosses itself, and since the user is playing a game, they might be okay with learning these things. Then recognition could be based in significant part on these purposely chosen features as well.