Leap Motion works!

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: Leap Motion works!

Post by MSat »

mahler wrote:
android78 wrote:The point is that, if you mount it on the goggles, the maximum distance it can see should reach the end s of your arms....
Based on DavidHolz' comment, I conclude it won't be very accurate for me.
3. What is the effective min and max range of the device? How much does accuracy fall off with distance?
Minimum range is about 1 cm, maximum advertised is 1 meter (we can do longer, but accuracy decreases).
Distance from my eyes to my the tip of my index-finger is around 110cm

The idea is awesome though... being able to see your hands that easily in VR
Like brantlew said "We really know almost nothing about the performance and limitations of this device yet."
I'll wait and see, but I am trying not to get my hopes up for this based on a few videos in a totally different setting.

Still really looking forward to this device.

To be fair, your arms are MUCH longer than the average person's. And while the Leap may not be able to track your fingers accurately at that distance, I'm assuming it should be able to track your arm just fine. At any rate, how likely is it that you would even need to track your fingers with your arms completely stretched out? Also, while the distance may be limited in the Leap Motion, I doubt it's inherent in the technology itself, so a version built for the consumer Rift may be able to accommodate a larger tracking area. If the Leap Motion tech works as good as advertised, it might just prove to be the most elegant solution for arms/hands/finger tracking. No gloves or anything! Just strap on the HMD and you're good to go!

User avatar
xhonzi
Binocular Vision CONFIRMED!
Posts: 273
Joined: Wed Mar 31, 2010 3:35 pm
Location: Thornton, CO USA

Re: Leap Motion works!

Post by xhonzi »

MSat wrote:
mahler wrote:Mounted correctly? My arms are over 3 feet long.
To be fair, your arms are MUCH longer than the average person's.
Mahler- are you 9 feet tall?

User avatar
Namielus
Certif-Eyable!
Posts: 957
Joined: Thu Aug 02, 2012 8:49 am
Location: Norway
Contact:

Re: Leap Motion works!

Post by Namielus »

sounds like abnormally long arms
Riftoholic

My precious 6 month project the Oculus Virtual Lounge:
Image
If you help me in any way I will be forever grateful.

User avatar
mahler
Sharp Eyed Eagle!
Posts: 401
Joined: Tue Aug 21, 2012 6:51 am

Re: Leap Motion works!

Post by mahler »

xhonzi wrote:
MSat wrote:To be fair, your arms are MUCH longer than the average person's.
Mahler- are you 9 feet tall?
Namielus wrote:sounds like abnormally long arms
Around 2 meters tall actually, but yeah... I'm a giant with long arms :roll:
The "110cm" in my following reply was more accurate.

User avatar
mahler
Sharp Eyed Eagle!
Posts: 401
Joined: Tue Aug 21, 2012 6:51 am

Re: Leap Motion works!

Post by mahler »

Some pictures have been released
Image
Source: http://www.monleap.fr/537-kit-developpe ... on-photos/

Developer kits from Leap Motion were sent out Nov 10th: https://twitter.com/LeapMotion/status/2 ... 4131850240
So keep an eye on the reactions: https://twitter.com/search?q=%23leapmotion

jf031
Cross Eyed!
Posts: 167
Joined: Fri Sep 28, 2012 4:32 pm

Re: Leap Motion works!

Post by jf031 »

First dev video of the Leap, tracking one finger:
[youtube]http://www.youtube.com/watch?v=H_X0iEP9kHI[/youtube]
Looks extremely accurate and without noticeable latency, as expected.

Found out about the video via a retweet from @LeapMotion. It looks like they'll retweet any dev video they get.

Edited to add: A better video to compare input to output:
[youtube]http://www.youtube.com/watch?v=SaoKy8bTgT8[/youtube]
Definitely some noticeable latency there, but likely the majority of it is due to the display. Very close to perceptually immediate, though.

STRZ
Certif-Eyed!
Posts: 559
Joined: Mon Dec 05, 2011 3:02 am
Location: Geekenhausen

Re: Leap Motion works!

Post by STRZ »

I hope that they don't get bought by Apple..

User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11402
Joined: Sat Apr 12, 2008 8:18 pm

Re: Leap Motion works!

Post by cybereality »

Thanks jf031, looks nice.

Balor
One Eyed Hopeful
Posts: 13
Joined: Mon Oct 29, 2012 12:20 pm

Re: Leap Motion works!

Post by Balor »

While Leap sounds nice in theory, what would that accomplish?
Of course, for virtual object manipulation this is fine. But some sort of input for movement, hotkeys, etc would be needed, and using gestures for that would be too tiring and slow.

For instance (unless you posses an unidirectional treadmill) how would you control moving your body in shooters? Some sort of wireless nunchuck-like controller?
Given that orientation would be tracked by Leap, having a couple hydra-like controllers (but smaller, cheaper, and wireless) should do the trick, but it would have to be custom-made for the rift I presume.

By the way! I've found out that you can actually convert wii nunchuck to USB. But still, it will not be wireless, and it only has two buttons.

blasphemer
One Eyed Hopeful
Posts: 13
Joined: Wed Aug 22, 2012 8:32 am

Re: Leap Motion works!

Post by blasphemer »

I imagine some future verion of the Leap will be integrated with Oculus Rift, tracking everything in your field of view. This will include hands, a gun, or some custom controller objects, depending on the game.

User avatar
Namielus
Certif-Eyable!
Posts: 957
Joined: Thu Aug 02, 2012 8:49 am
Location: Norway
Contact:

Re: Leap Motion works!

Post by Namielus »

You can also have one located at your feet tracking something like the wizdish (affordable omnidirectional treadmill) or simply walking on the spot
Riftoholic

My precious 6 month project the Oculus Virtual Lounge:
Image
If you help me in any way I will be forever grateful.

User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Leap Motion works!

Post by FingerFlinger »

With regards to mounting the Leap to an HMD, there was a discussion just the other day on the VR Geeks mailing list. Virtual objection manipulation will work, but there are constraints. First, the user must be looking at the object the entire time, or the Leap will not be able to track the hand motions. This is very different from the way that humans normally manipulate objects; tactile feedback is so ingrained that most of the time, we (as humans) use vision to address our ourselves to objects and then look away quite often, if not the majority of the time.

Also, the main contention of this discussion is that mapping the user's hands to the virtual environment will be more difficult than it seems, because it is necessitated on good full-featured head tracking. And even then, dealing with a wildly changing origin (the head tracked HMD) may still not be trivial.

I personally think it is still the best use-case, but it won't be plug-and-play, by any means.

User avatar
Namielus
Certif-Eyable!
Posts: 957
Joined: Thu Aug 02, 2012 8:49 am
Location: Norway
Contact:

Re: Leap Motion works!

Post by Namielus »

I agree and I think chest mounted makes more sense.

Now that they are out tho, its going to stop being theoretical and all these things can be tested.
Riftoholic

My precious 6 month project the Oculus Virtual Lounge:
Image
If you help me in any way I will be forever grateful.

User avatar
mahler
Sharp Eyed Eagle!
Posts: 401
Joined: Tue Aug 21, 2012 6:51 am

Re: Leap Motion works!

Post by mahler »

FingerFlinger wrote:Also, the main contention of this discussion is that mapping the user's hands to the virtual environment will be more difficult than it seems, because it is necessitated on good full-featured head tracking. And even then, dealing with a wildly changing origin (the head tracked HMD) may still not be trivial. I personally think it is still the best use-case, but it won't be plug-and-play, by any means.
As said in that thread: Just displaying the arms/hands relative to the POV might be nice for now.

EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: Leap Motion works!

Post by EdZ »

FingerFlinger wrote:This is very different from the way that humans normally manipulate objects; tactile feedback is so ingrained that most of the time, we (as humans) use vision to address our ourselves to objects and then look away quite often, if not the majority of the time.
Without some pretty heavy-duty haptic glove hardware (which by necessity will have joint-tracking built in), you won't be doing any sort of unseen tactile manipulation with the Leap Motion, Kinect, stereo cameras, or whathaveyou. When you're not looking directly at your hands, you have zero feedback whatsoever as to the state of the objects you're manipulating, and only proprioceptive feedback of the state and position of your hands.
And this proprioceptive feedback isn't very good. As a demonstration, form some sort of shadow-puppet-like shape with your hands in front of you, without your hands touching each other. Now, close your eyes. Put your arms straight down by your sides, touch them behind your back, touch them at the back of your head, then remake the shape in front of you. Then open your eyes: I can guarantee you will either have hit your hands together when reforming the shape, or you will have them out of alignment.

Visual feedback will generally override proprioceptive feedback in the human brain (incidentally, this is why mirror therapy can alleviate some kinds of phantom limb pain). I'm not going to advocate deliberately cutting off blood flow to your limbs, but next time you get 'pins and needles' (or a 'dead leg' or whatever your local phrase is), see how well you can move it when looking at it, and with your eyes closed. The exception to this is touch, which is a very powerful and detailed sense in your fingertips.

Aside from the difficulty in tracking with two degrees of separation (hands->head->torso(->world)), mounting your sensors to the front of the RIFT has the benefit of maximising the area your hands can be tracked while inside your field of view. With a chest-mount, your could not manipulate objects above you, for example. As you already cannot effectively manipulate virtual objects with your hands if you cannot see them, making the two congruent is fairly efficient and embeds the 'see to interact' relation in the player's mind.

User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Leap Motion works!

Post by FingerFlinger »

@mahler

Yeah, that was me saying that. Certainly it is the first step, but as for going beyond that point, we really need to be able to play with and actually develop for the actual devices. Theory becomes pretty fuzzy when it comes to what will work in a UI scenario and what will or will not be immersive.

@Edz

I agree. The only reason I brought it up is that I feel that without tactile feedback, we are probably limited to gestural-type actions from the Leap, which are easy to do without staring at your hands. Stuff like waving troops forward in an RTS, or snapping your fingers, or pointing. These are all natural actions that could increase immersion, but I usually don't look at my hands when performing them.

EDIT: I guess I left way too much to inference in my first post

Chrysophrase
One Eyed Hopeful
Posts: 4
Joined: Wed Oct 10, 2012 5:04 pm

Re: Leap Motion works!

Post by Chrysophrase »

Just saw this... Unity and leap motion integration for hand tracking. Pretty impressive for being so early in the dev process.

[youtube-hd]http://www.youtube.com/watch?v=6_9vr3FQ0Zc[/youtube-hd]

zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Leap Motion works!

Post by zalo »

Accepted. :D

DavidHolz
One Eyed Hopeful
Posts: 2
Joined: Wed Aug 01, 2012 12:18 am

Re: Leap Motion works!

Post by DavidHolz »

I've personally always wanted to be able to shoot two different guns (ie: machine pistols ala Matrix style) simultaneously in different directions. Or preform swat team squad commands in a seamless/silent manner. Meleeing an enemy might be cool too :).

The first time I put a Leap on a HMD and combined it with our stereoscopic point-cloud visualization I was floored by how strongly and how easily phantom-limb syndrome sets in. Regardless of what my rational mind tells me (that a collection of points was not my hand) the rest my mind was totally, completely, adamantly, certain that those fuzzy points WERE MY HANDS. When this happens any sort of audio/visual feedback can go a long way (since your mind is so convinced that it's real the rest of your physical-coordination wetware kicks in fully to remove any further disabilities).

However, that said, it's still very strange when you clap/touch your hands together (since you'll see two virtual hands touch and you'll suddenly have perfect haptics for a brief moment).

Without a HMD as well, physical-type interaction can leave a pretty strong impression that the digital thing you were doing was (against all logical reasoning) a true physical experience. It's similar to the visceral aspects of touch based devices.

Just my two cents on some of the things being discussed :mrgreen:.

Flassan
Cross Eyed!
Posts: 151
Joined: Wed Apr 18, 2012 3:27 pm
Contact:

Re: Leap Motion works!

Post by Flassan »

That's really interesting David. Taking what you said about haptics to what extent is the illusion ruined if you hold something generic and abstract, such as a Move or WiiMote? For instance, if I'm holding a portal gun when I look down my hands would not be in the right position. You may not spend much time doing that but I'm interested to know if you've tried it and what your impression was.
Also, how well would the Leap work for full body tracking if the person stays in the same place? I realise this reduces the accuracy but by how much?

MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: Leap Motion works!

Post by MSat »

There is a huge dependence on the way we intend to interact with the virtual world. Unfortunately, there's no single solution, so it varies greatly between different kinds of content.

The benefit I see in using the Leap Motion always facing your POV is accurately tracking your arms/hands/fingers/objects when they're in your view, adding greatly to the sense of immersion, while using minimal amounts equipment. Generally, requirements for such accuracy diminishes when no longer in view, and can be achieved with other methods such as "9DOF" sensors in a gun/controller, and maybe wrist/glove finger trackers if gestures are needed.

I am very curious about the Motion's capability to track body skeletal data from a location such as the bottom of an HMD. There seems to be a huge need for detecting head translations and orientation at least relative to the body. Do you have any info you could share with us about this, David?

zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Leap Motion works!

Post by zalo »

The leap SDK is really easy to get started with. Received my dev unit today.

[youtube]http://www.youtube.com/watch?v=Y4J7AsTI5vk[/youtube]

I'll get a real video up later when I have a USB 3.0 cord.

druidsbane
Binocular Vision CONFIRMED!
Posts: 237
Joined: Thu Jun 07, 2012 8:40 am
Location: New York
Contact:

Re: Leap Motion works!

Post by druidsbane »

zalo wrote:The leap SDK is really easy to get started with. Received my dev unit today.

[youtube]http://www.youtube.com/watch?v=Y4J7AsTI5vk[/youtube]

I'll get a real video up later when I have a USB 3.0 cord.
Is there a Linux SDK? I wanted to get Leap set up for my VR desktop Ibex but I haven't been accepted yet and I think I have to wait till Feb for my preorder to arrive:( Hopefully no delays! Pretty excited!
Ibex 3D VR Desktop for the Oculus Rift: http://hwahba.com/ibex - https://bitbucket.org/druidsbane/ibex

STRZ
Certif-Eyed!
Posts: 559
Joined: Mon Dec 05, 2011 3:02 am
Location: Geekenhausen

Re: Leap Motion works!

Post by STRZ »

druidsbane wrote: Is there a Linux SDK? I wanted to get Leap set up for my VR desktop Ibex but I haven't been accepted yet and I think I have to wait till Feb for my preorder to arrive:( Hopefully no delays! Pretty excited!
Not yet, i read in their forums that they're considering Linux support. But the driver wouldn't be open source though. I hope that they open source the part of the driver wich doesn't expose their intellectual property wich "does the trick". So that people could use it to design new native window managers and desktop environments wich take advantage of it. Linux is far more useful for this than Mac or Windows, because you can replace the default desktop entirely. From a technical viewpoint it would be really stupid if they don't support Linux.

Imagine a Linux distro specialized for this stuff, the Rift, Steam etc. and not depending on Apple or Microsoft and their politics torwards "normal users".

zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Leap Motion works!

Post by zalo »

If and when the Linux SDK is released, I'm sure that at the bare minimum, you'll have access to the fingertip positions relative to the leap.

I also pre-ordered the consumer leap because the final hardware will be several times better than the dev unit (though the dev unit already does absurdly precise positioning at a very low latency anyway).

Personally, I'm also excited for the leap being a super precise replacement for TrackIR that can get 3D position from just one IR reflector! At its core, the leap is really two high resolution infrared-only cameras with insane FOVs and insane refresh rates (295hz with USB 3.0). Plus, it has self contained IR illuminators at equally high FOVs. When it's released, it'll be everything the PSEye could never be for consumer point tracking.

Coupled with the Rift's absolute orientation, it could be the inexpensive solution to the positioning problem. Simply mount a leap on the ceiling above you, stick an IR reflector sticker on the top of your headphones, and code the one-to-one interface. Perfect for sitting down, walking in place, and using WizDish scenario.

Post Reply

Return to “General VR/AR Discussion”