Rift-Mounted Leap Motion Controller Demo (WIP 4: 5/9/13)

[zalo]

Update:
https://www.youtube.com/watch?v=zGdkLOXKS04
This is a much better view using Fraps that you can view with your rift. It demonstrates two new melee weapon types and a preliminary menu system.
--------------------------------------------------------------------------------------------
Update:
[youtube-hd]http://youtube.com/watch?v=7BRRx7MelHQ[/youtube-hd]
Here I added a little sword to play around with, fixed the physics, and shaved down the entire script to 29 beautiful lines.
Here I show off how much fidelity and spatial awareness you can get by moving the "sword" all around the object and tapping it on each side.
--------------------------------------------------------------------------------------------
I made a little demo to show off how well the Leap does hand tracking when mounted on the front of your rift.

http://www.youtube.com/watch?v=Mk8guY5W_ww

http://www.youtube.com/watch?v=KQ6lTeW-_Ws

To use:

• Firmly attach your Leap to the center of the faceplate of your rift (I used velcro).
  If your hand is upside-down, go into the Leap Tray Icon and click "Reverse Orientation".

Download Here

Do you think it's a viable control scheme?

[brantlew]

I would love to see a quick video clip.

[mattyeatsmatts]

he's got one on youtube I saw it the other day, just doesnt look like it works very well but I would like to see the leap used for position tracking with the leap attached to your head with a piece of string with some sort of receptor dangling from the ceiling directly above your head.

[zalo]

This isn't on YouTube, I finished it last night. This one is a hand tracking demo in the style of the hydra, the other is head tracking that was ultimately a pain in the butt to get working with the Rift camera.

I'm probably going to make a video later today if someone else doesn't do it first.

[zalo]

Updated OP with video. It also demonstrates how head tracking and hand tracking work together... or should I say: "Go hand in hand"

[Visual Knight]

This looks so much better than the hydra- your not holding anything- and you see your hands AND FINGERS move- This is mind blowing. Now- if someone could make gloves that vibrate on the touch of surfaces for tactile response- athere maybe a way of enhancing the sensors with gloves and a mounted camera

[cloudier]

This is exactly what i have been waiting for, i have 3pcs of leap motion ordered.

Hopefully someone will make a razer hydra emulator for the leap motion, so it can be used in games.

Qustion, do leap motion automatically keep track of where the leap motion are positioned? I thought that if you moved your head, the hands in the game would move too but it does not seem that way.

[yoshithedog]

Far, far better than the Hydra. This is the future.
Would be nice if The Gallery and other games got Leap support.
Now we only need good, cheap and simple feet movement tech.

[mickman]

Wow nice to see some Leap Motion action happening on the RIFT ... If I had my Rift I'd be doing the same

The hardest part I found using the Leap was getting a feel of proximity.. exactly where an object lies in space...

That's a great idea to simply attach it to the front of the RIFT. Its so light you'd never feel it.

regarding LeapMotion for sale: hhmm... Well that sort of sux ...I thought it would be similar to the Rift .. I notice a few for grabs on e-bay...

http://www.youtube.com/watch?v=TMqBLqzsZZU

[LordJuanlo]

This looks great! Want it!

[Paladia]

The problem is the narrow field of view. How long do you reckon one can keep the hands moving in front of ones face before getting tired? Probably not too long.

[GeraldT]

FOR SALE - ONE LEAP MOTION

You got yours for free! Also yours is bound to your developer ID so the potential buyer faces all sorts of trouble.

If you are a developer enter the developer program on Leap Motion. Since they delay the start they send out more dev kits. Don't buy them from unreliable sources. The dev kit needs an online connection and is bound to the registered developer.

[zalo]

In addition to it being linked to your account, you also made an agreement with them that you wouldn't sell it. But what is someone's word worth these days? If a mod would be so kind, I would appreciate it if you could take down that post.

Also:
http://www.youtube.com/watch?v=Mk8guY5W_ww

When I ordered the dev kits (both), I was worried I wasn't developer enough to do anything nifty with them.

[usb247]

The problem is the narrow field of view. How long do you reckon one can keep the hands moving in front of ones face before getting tired? Probably not too long.

I don't know, from the video it looks like the hand movement fills your entire FOV.

[Lynkk]

It's going to be perfect for flight simulators such as FSX or DCS where you can click on the VC.

[mickman]

Biggest issue with the Leap is the fact its so darn hard to get a true feel of spatial distance... as seen above, you tend to go straight through objects.

So far it seems fine for controlling a vehicle in 3PP but try reaching out to change a gear stick in 1PP and you find out its so hard to actually make contact.

Tapping large interface buttons, swiping through a menu, enlarging images or tap for a way point work well... but object manipulation is darn right difficult.

[Aabel]

Very nice work!

Can you use multiple leaps to increase the capture field? or will each leap report it's own tracking? I've heard a lot of negatives about the leap SDK but this video is a pretty strong positive!

[zalo]

Biggest issue with the Leap is the fact its so darn hard to get a true feel of spatial distance... as seen above, you tend to go straight through objects.

Spatial maneuvering is actually pretty intuitive. I circled my hand around the cube without touching it a couple of times to show how good spatial awareness is. The stereoscopy really helps. The collision is terrible because I literally glued some scaled cubes to each finger and the palm. Razer hasn't released the source code for its ACTUAL Oculus/Hydra demo, so grabbing pretty much has to be done DIY.

To tell you the truth, reaching to change a gearshift in a car will probably work much better than navigating a menu. The hand's position is quite jumpy, so you'll probably hit the wrong menu item. Jumpy-ness is more forgiving when you're reaching out to an object that only needs an approximate position and has no other objects near by (it can kind of "lock on" to the gear shift). And it's only really jumpy because of the awkward angle. If I had it on the table, menus would be more intuitive, but your range of motion is now limited to in-front of you and not where you're looking. So now manipulating objects is difficult. Trade-offs.

@Aabel Leap Motion has said that they will add support for multiple leaps (and I have two; they're different models though) so if they do that and fuse the hands and positions into one capture volume, I definitely want to try it.

[zalo]

Added a new video to the OP!

Physically responsive swordplay. Or blunt-stick play. Whatever. It's still just as fun!

[MrGreen]

Impressive stuff zalo!

[Laserschwert]

I notice a bit of lag between your motions and the screen... is that actual Leap lag, or is your monitor just lagging behind what the Rift sees?

[zalo]

UPDATE:
https://www.youtube.com/watch?v=zGdkLOXKS04
Works great to view on your rift and in free-view! Two new melee types and a preliminary menu system.

@Laserschwert
The lag in some of the videos is because I set the leap to "Precision" as opposed to "High Speed". In High Speed, there is little noticeable lag and no appreciable loss in precision. Though you can't really see it, I did this latest video in High Speed.

[dbuckvs]

thanks for the code snips It's probably time to start dusting off the leap here!

nice demo, it must be creepy as hell to beat up a ragdoll while 'in-rift'!

[LoTekK]

I've been experimenting with doing similar, and came across your thread. I've tried with both my Leap Motion and Creative Gesture Camera, but the key thing I've having trouble with is, and it's a big one, signal noise. I get crazy false positives all over the place, no matter where the camera is looking. Are you doing anything in particular to deal with signal noise, or are you just lucky in that you're not hitting any sort of false positives?

[zalo]

That's a great question. For me, I can mitigate false positives by looking towards an empty open space (by turning around in my swivel chair). It doesn't really matter what direction you're looking in unless you need a keyboard, which this new input enables you to discard.

I'd also avoid turning towards windows and bright sunlight. Not ideal solutions, but they're a start (and David mentioned future sdk versions will increase environmental robustness, so let's hope that helps).

[LoTekK]

Ah, that's a little unfortunate, was hoping to have a bit more control over noise reduction. Soon, hopefully! In the meantime, I should have a Razer Hydra getting delivered before too long, which looks like a pretty solid complement to the Rift (especially since you can still do stuff out of view)

[BeeX]

It looks good.
When I first thought of this integration I had the doubt that it wouldn't be that good because of the distance between Leap and the max point of the stretched hands, but you get a fair interactive space of fov..

[Freakling]

Awesome stuff, I've got my Leap but I'm still waiting for my Oculus Rift.
I will definitely test this when everything gets here.

On another note I've been wondering if this can be used to provide Verical, Horizontal and depth to the head tracking.

[zalo]

Leap appears to be adding something special to their SDK soon:

[MattStompz]

It's been a while since someone posted on this forum, but I recently got a Leap Motion and have mounted it to the Rift. I'm working on development for it but it is difficult. Seems it is not as simple as adding colliders to the hands and objects they will interact with. It all must be scripted (seems a little non-standard for script too).

This demo was actually the smoothest demo I've seen for the LM. The fingers are always an issue, but the palms/hands are captured wonderfully! How did you do it?

[BloxRift]

Also ive notice by placing at a 30 to about 45 degree angle downwards you dont have to keep your arms as high

[LM_Edwin]

Hello MTBS3D Community! I'm Edwin, Community Relations Project Lead at Leap. I'm here to answer any questions you may have and share cool/interesting AR and VR projects that we find. I hope you guys have checked out the George Takei Explode a VR Shark video (https://www.youtube.com/watch?v=StShniPEXmU), it's pretty sweet! Is anyone working on an awesome project they want to share?

[cybereality]

Edwin, if you haven't spoken to Neil yet, you should send him a PM:
http://www.mtbs3d.com/phpBB/ucp.php?i=p ... ompose&u=2

There are certain rules regarding commercial entities on the forum. Thanks.

[Neil]

Ah! It's Edwin from Leap Motion! We've actually featured your stuff on MTBS regularly; very exciting gesture tech.

https://www.youtube.com/watch?v=AXxo7iMGPco

Cybereality is right that this isn't the best channel, though I know the members are excited about your work and with good reason. Let's reach out privately and figure out the best method to interact with the community on MTBS. Stuff like this happens all the time; don't sweat it.

Regards,
Neil