Limited hand tracking in conjunction with HMD-mounted Leap?

Discussion of tools and products that add VR physicality. Samples include VR treadmills, special hand controllers, gesture technology and more.
Post Reply
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Limited hand tracking in conjunction with HMD-mounted Leap?

Post by MSat »

More accurate thread title would have been "Thought's on limited hand tracking in conjunction with HMD-mounted Leap Motion-like systems"

This thought has been in back of my mind for a little while but I figured with the growing VR related hardware market it would have been mentioned by now. Many people here as well as the Leap Motion guys immediately saw the possibilities to having a Leap Motion (or something similar) mounted to an HMD for the purpose of hand tracking (but not just limited to it, of course). In theory, this would work great as long as your hand remains within the optical tracking system's FoV, but that might not always be the case, and in some instances it can be problematic. For example, say you're playing an FPS and have a pistol style controller in hand. While it's within the tracker's FoV everything is fine, but what happens if you're shooting at an object and quickly glance somewhere else so that your hand is no longer being optically tracked? The software could make the assumption that your hand hasn't moved since it was last tracked, but that's just a hack. Of course, there's other motion tracking hardware such as that from Sixense and others, but they're a bit pricey and they make optical systems that I believe are likely to be incorporated in future HMDs to be redundant. One of the cheapest, and relatively most robust methods would seem to be a controller with an IMU. You don't get absolute positioning, but you do get orientation which would be sufficient for most scenarios, and when your hand is within your FoV you see it in the proper position because of the optical tracking system. The good thing is that there are already numerous such devices on the market, but the bad thing is they cost more than they should.

If Oculus is going to have such optical tracking on their consumer HMD, I hope that they consider offering such controllers as a low-cost option to keep VR hardware as affordable as possible.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by zalo »

http://www.youtube.com/watch?v=KQ6lTeW-_Ws

http://www.youtube.com/watch?v=Mk8guY5W_ww

http://www.youtube.com/watch?v=7BRRx7MelHQ

http://www.youtube.com/watch?v=zGdkLOXKS04
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by MSat »

Is this your work, zalo?
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by zalo »

Yeah, It's something I've been toying around with in my spare time.

However, it looks like Leap Motion's Tracking Fidelity is about to reach a whole new level (~20 seconds in):
[youtube-hd]http://youtube.com/watch?v=hGL0ZFBbY8M[/youtube-hd]

I'm excited to fart around with some skeletal tracking :D

EDIT: .gif form for those who want to see it more closely:
Image
Last edited by zalo on Wed Nov 20, 2013 11:34 pm, edited 1 time in total.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by cybereality »

Best Buy? Good for them.

May have to pick one of these up.
thenomad
One Eyed Hopeful
Posts: 22
Joined: Tue Jul 02, 2013 9:29 am

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by thenomad »

How is the Leap Motion's tracking coming along these days? I recall they got some pretty bad reviews after launch: were those first-day jitters or did it generally not meet expectations?
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by cybereality »

I tried it and I was not particularly impressed. I mean, it did work within a small area and with limited gestures.

For some applications this may be OK, but I don't think it's quite enough to do full hand tracking like you might want for VR.
User avatar
GeraldT
Certif-Eyable!
Posts: 1057
Joined: Fri Jan 18, 2013 9:10 am
Location: Germany
Contact:

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by GeraldT »

have you tried a Rift mounted version yet Cyber?
Once you mount it to the Rift the distance between hands and camera becomes bigger and so does the tracking area. Still not a perfect solution, but a cheap one.
If they manage to get the full skeletal tracking working as seen in that video there might be a lot more potential to it.
want to demo the Rift or check it out? click here
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by cybereality »

No, I just had it on my desk. I agree there is a lot of potential with the technology, but it doesn't seem optimized for VR at this point.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by FingerFlinger »

Part of the issue right now with a Rift-mounted Leap is the lack of positional tracking. Without 6DOF tracking, there are two separate frames-of-reference for the Leap and the head position. The simulation would not be able to distinguish between the hand moving closer to the HMD from the user leaning forward closer to his/her hand. This would manifest itself as the hand changing scale in an unexpected manner. With position tracking, this problem becomes tractable, but it must be high precision AND high accuracy. Something like the current Hydra would not be good enough (I suspect).
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by zalo »

The problem is bigger with a desk mounted leap and no positional tracking; the mismatch is insane.

With the rift mounted leap, the view and device exist in the same frame of reference; the virtual hand matches up with the proprioceptive position of where you expect your hand to be, so it works beautifully.
I've been experimenting with this a lot more lately, and I've decided that a rift mounted leap is much preferable to a desk mounted leap in VR (plus you get to move your hand in the 360 degree space around you (where you're looking)).


Also, I'm not aloud to say a whole lot about the upcoming Skeletal Tracking API, but I think I should mention that it works very nicely for Rift-Mounted Experiences :)
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Limited hand tracking in conjunction with HMD-mounted Le

Post by FingerFlinger »

That's good to hear. I'm glad that you're still working on this!
Post Reply

Return to “Physical VR Tools”