From fels@mic.atr.co.jp Mon Jun 29 00:56:12 1998 Received: from burdell.cc.gatech.edu (root@burdell.cc.gatech.edu [130.207.3.207]) by lennon.cc.gatech.edu (8.8.4/8.6.9) with ESMTP id AAA19817 for ; Mon, 29 Jun 1998 00:56:11 -0400 (EDT) Received: from wheaten.hitl.washington.edu (itqukgtTvJm0a44gIeKkdlCb2MzgxzMv@[128.95.73.60]) by burdell.cc.gatech.edu (8.8.4/8.6.9) with ESMTP id AAA02427 for ; Mon, 29 Jun 1998 00:56:06 -0400 (EDT) Received: from mailhost.mic.atr.co.jp (mic.atr.co.jp [133.186.20.201]) by wheaten.hitl.washington.edu (8.8.8/8.6.12) with ESMTP id VAA25181 for <3d-ui@hitl.washington.edu>; Sun, 28 Jun 1998 21:55:44 -0700 (PDT) Received: from miris43.mic.atr.co.jp by mailhost.mic.atr.co.jp (8.9.0+3.0W/3.7W) id NAA22569; Mon, 29 Jun 1998 13:55:13 +0900 (JST) Received: by miris43.mic.atr.co.jp (8.8.5+2.7Wbeta5/3.5Wpl709/11/97) id NAA18173; Mon, 29 Jun 1998 13:55:12 +0900 (JST) From: "Sidney.Fels" Message-Id: <9806291355.ZM18171@miris43> Date: Mon, 29 Jun 1998 13:55:12 +0900 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: 3d-ui@hitl.washington.edu Subject: glove-talkII discussion plus more Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO Hello all: I sent this message to Chris who suggested I forward it along to the list. I'm newly on the list so I haven't had a chance to go back over the archive to see what's been said so I hope some of the comments below aren't redundant. If you have any comments please send them off to me. thanks. sid Here's the message: > Hi Chris, > > How are things? > > Here's the simple explaination: the idea behind GloveTalk2 was that I had the > speaker instrumented with a CyberGlove, a polhemus sensor, a contact glove and > a foot pedal. I then provided a mental model of a vocal tract-like space to > the > speaker. They then went through training sessions where a sound was played for > them and they would make the gesture corresponding to that sound (using the > mental model). The training data collected from the speaker was used to > adapted the mapping between the input devices and the speech synthesizer. > After 100 hours of practise and about 15 iterations of training he could speak > with the device. > > Here's the abstract from one of my papers: > > Glove-TalkII is system which has an adaptive interface built with neural > networks. Glove-TalkII maps hand gestures continuously to 10 control > parameters of a parallel formant speech synthesizer. The mapping > allows the hand to act as an artificial vocal tract that produces > speech in real time giving an unlimited vocabulary and unlimited control > of fundamental frequency and volume. The best version of Glove-TalkII uses > several input devices (including a CyberGlove, a ContactGlove, a 3-space > tracker, and a foot-pedal), a parallel formant speech synthesizer and 3 neural > networks. One subject has trained to speak intelligibly with Glove-TalkII. He > speaks slowly with speech quality similar to a text-to-speech synthesizer but > with far more natural-sounding pitch variations. > > There is also a video available from UofT if you want it. > > I briefly read your "gloves suck" letter. I didn't have time to read the whole > thing, but, I essentially agree. Though,I also would make the comment "glasses > suck" but we still wear them. If an application was out there that really made > my life better I'd be happy to wear gloves. For example, when I'm skiing and > it is cold I'd rather wear gloves than not. However, currently no such > application > really exists (though I'm working on one now) so gloves don't seem to be very > useful. As you suggest, most applications are nothing more than a polhemus > with some buttons. This direction makes sense in the 3D computer graphics > world since most of the interaction is dealing with spatialization parameters > of objects. To move beyond that, i.e., to think about object qualities (even > deformations) requires thinking about objects abstracted away from cartesian > spaces which is where Polhemuses with buttons won't necessarily work. It is > for this reason, that my own research with gloves goes directly away from using > gesture recognition (which is button pressing) and relies only upon topology > mappings of continuous spaces. These applications are difficult to find in the > computer world but are very interesting and useful in everyday life, I think. > > As for vision based input - that's a whole different topic. Using as a > replacement for the polhemus is welcome. Using it for gesture recognition is > identical to using a glove in the sense that you are just adding buttons to a > tracker - it hasn't really solved anything. I think that we need to consider > what gesture is really good for first, then worry about making the best input > device to do the job. Instead, what happens is here's something that's hard to > do with the mouse - let's do it with gloves and trackers. > > I think that one way to really help to find applications for gloves that would > make them seem less of a bother would be to make them wireless. At least then > you would not have the encumberance of a wire. I know from working at Virtual > Tech. (2 years ago) that we talked alot about making the glove wireless, > but, the final consensus was, that until Polhemus makes the tracker wireless, > Virtual Technologies won't either. I don't know if that is still their > opinion, but it looks like it is. > > When I get a few moments, I'll try and re-read the input thread on 3dui so I > can make more coherent comments. > > I'll be back at UBC in Sept. so I hope well be able to get together at some > point in the near future and talk about Canadian research scene and see if > we can do something together. > > Hope you are doing well. > > sid >