New Microsoft Research Hand Gesture Tracking Prototype

[Bretspot]

http://www.engadget.com/2012/10/09/micr ... -tracking/

Really Cool! It doesn't look like it can really track elbows well though, maybe Kinect 2.0?

[brantlew]

Wow, wow, WOW!! That is brilliant and amazing. I wonder how much that can be miniaturized because if it could be made the size of a watch that would be the best interface device I can imagine. Go Microsoft! I just wish they had a better track record of turning ideas into commercial products.

[gravity360]

I'd like to use a dual setup like this in combination to sit down at a virutal desktop and control my system as normal. Basically the desktop would be a 3D interactive environment with a virtual pc. Maybe I should setup my pc on a Hyper-V system! Combine this with a kinect and you could have a very interesting setup. But yes there's a ton of potential with this!

And yes, I too hope this doesn't turn into another project like "Courier" I really wish that would have came to life, I was so prepared to run out and buy it as soon as I could. Maybe that should become the next kickstarter project

[brantlew]

I wonder if the Leap system could be modified to do this? It's already pretty small but it might require some customization to track hands in this orientation. It's unclear how much the Leap employs bio-mechanical hand modeling.

[gravity360]

True, from what I recall the Leap systems tracking was outstanding, but how does it scale on coverage area as well? I'll have to go and dig up more information on it's inner workings.

[wuliheron]

This is exactly the kind of thing I was hoping someone would produce for the Oculus and the fact it is a prototype made with cheap off the shelf components is even more impressive. Microsoft has a pretty good track record for producing quality low cost peripherals too.

Even more interesting is while people have been complaining about Windows 8 it's been obvious to me from the start Microsoft's intention wasn't just to create a new interface for portable devices. They've been doing this kind of research for years and their labs indicate a combination of keyboard, gui, gesture, touch, and voice commands is the most intuitive and fastest interface possible. It may sound complicated, but the idea is you use them in any combination that appeals to you. A three year old might prefer to touch the screen more often, a senior citizen to use voice commands, or whatever. This kind of controller could replace the mouse.

[android78]

...the fact it is a prototype made with cheap off the shelf components is even more impressive...

It looks to me like the 'cheap off the shelf components' was essentially they got a kinect, took the IR pattern projector and camera out of it, probably changed the lens in the projector, and strapped it to their wrist.

[WiredEarp]

Wow, wow, WOW!! That is brilliant and amazing

Maybe im just a bit jaded, and it looks great, but not exactly brilliant, more like an obvious evolution. Its no different in concept to putting a Leep on the wrist, along with a gyro tracker, is it?

I wouldn't be surprised if they try to patent it next...

[gravity360]

They should., its a really nice concept. I think the elbow tracking issue could be resolved with a simple fix. I just hope That it can be made more compact like their claiming.

[WiredEarp]

@ gravity360: I dont really think this sort of thing should be able to be patented (not saying it can't/won't be). Basically its just the same as mounting a kinect/Leep to your hand with a sensor. If this is patentable, can we just stick a Kinect on our heads, feet, ankles, etc, and also patent that? I mean Project Reality has the Hydra base unit on the head, is that novel enough to patent? What about since they have combined that with a gyro/PS3 tracker?

[wuliheron]

@ gravity360: I dont really think this sort of thing should be able to be patented (not saying it can't/won't be). Basically its just the same as mounting a kinect/Leep to your hand with a sensor. If this is patentable, can we just stick a Kinect on our heads, feet, ankles, etc, and also patent that? I mean Project Reality has the Hydra base unit on the head, is that novel enough to patent? What about since they have combined that with a gyro/PS3 tracker?

Here's a more technical breakdown:

http://www.sciencedaily.com/releases/20 ... 093034.htm

The difference is like the Kinect they took a difficult set of algorithms and broke them down to a simplified set. Then they went a step further and made it entirely self-contained so you can use it with any device including a cellphone or TV like a universal remote control.

[cybereality]

I had this same idea a while ago, but figured people would not want a huge block of a thing on their wrist.

But still, great job that they got this working.

[Additives]

It doesn't look like it can really track elbows well though

if you could track the position of the device, you could feed it into an overall skeletal model. You can determine the positions of the whole arm (for fixed shoulder) from the position and angle of the wrist relative to the head.

E.g. If you were to combine these with the rift by adding a small IR camera to the front of the HMD to track 2 IR LED's on the wrist strap. By knowing the relative position and angle of the wrist to the front of the Rift (i.e. relative to your head), you can calculate THE ONLY possible position your arm could be in. Might need more than that in practice, as the calculations for inverse kinematics can be NP, and thus costly.

[gravity360]

It doesn't look like it can really track elbows well though

if you could track the position of the device, you could feed it into an overall skeletal model. You can determine the positions of the whole arm (for fixed shoulder) from the position and angle of the wrist relative to the head.

E.g. If you were to combine these with the rift by adding a small IR camera to the front of the HMD to track 2 IR LED's on the wrist strap. By knowing the relative position and angle of the wrist to the front of the Rift (i.e. relative to your head), you can calculate THE ONLY possible position your arm could be in. Might need more than that in practice, as the calculations for inverse kinematics can be NP, and thus costly.

I'd think that you'd want to go with what M$ has presented here and combine it with the hydra's tracking. The technology is there, just got to meld them together. Razer's Hydra is actually a really good tracking system. I think it would just give a more natural feel to the user interaction.

[gravity360]

You know, now that I think of it, they could shrink it down and use different a custom lens to do the trick. Hmm need to find my kinect and see what I can do... might need to get a second one.