ControlVR - Full upper body and finger tracking

Discussion of tools and products that add VR physicality. Samples include VR treadmills, special hand controllers, gesture technology and more.
Post Reply
BrandonJLa
One Eyed Hopeful
Posts: 1
Joined: Wed May 21, 2014 6:42 pm

ControlVR - Full upper body and finger tracking

Post by BrandonJLa »

Hey guys, I'm Brandon Laatsch, I've been lurking for a long time but am posting for the first time today.

Awhile back I was approached by a few guys about using their inertial motion tracking tech for the purpose of VR. For the first time, they had gotten IMU sensors small enough to place on each finger and for the past few years they have been creating 20 - 50 thousand dollar solutions for mo-cap, robotics, physical therapy, etc... With the advent of VR and the price of sensors coming down, we feel that for the first time it is possible to get this tech into/onto everyone's hands. Thus, by combing the efforts of Synertial (for the motion tracking tech) and Inertial Labs (for mass IMU production) and myself plus a few others, we're nearing being able to bring the systems to the public!

Below is a video of the system working with an Oculus DK1. It features 19 IMU sensors working in a FK system to capture the upper body down to the fingers with 6DOF. One is on your chest, two on the upper arms, two on the forearms, two on the hands, four on the thumbs (2 on each), and eight on the remaining fingers. The sensors are accurate to a tenth of a degree and work great, we don't have to "smooth" the data at all resulting in a responsive and reliable track. Also, by rooting the system with the biped's pelvis and using the chest sensor to drive the upper body as well as using the Oculus' sensor to drive the head, we're able to give 6DOF to the DK1.

On the more fun side, having spent some serious time with my hands and fingers in VR, I can testify that the sensation of seeing your hands and fingers move accurately to reality is really special. The relationship and familiarity that we have with our hands is really powerful. I'm not a psychologist, but I'd theorize that our hands movement is more important to our sense of "self" then anything else due to the amount of time we spend seeing and using our hands.

The plan is to launch a Kickstarter within a month and deliver the dev kit by Christmas this year.

We'll be demoing the gloves at E3 on June 10th - June 12th, but I'm also thinking about holding an event at my office in Downtown LA in the next few weeks so that anyone local could come over and see it live, let me know if you would be interested / able to come over some evening.

I'd love to here your feedback as I continue to develop this tech.

https://www.youtube.com/watch?v=LPszKhewSec

dokworm
One Eyed Hopeful
Posts: 5
Joined: Wed Jan 20, 2010 7:29 pm

Re: ControlVR - Full upper body and finger tracking

Post by dokworm »

Amazing, so much better than my old powerglove :)

Any ballpark as to pricing on this? We are developing a VR and Dome product to help people with physical and cognitive disabilities, the extra immersion that having fully functional hands in the simulation would be ideal. I am a psychologist and you are spot-on with your comment about the importance of seeing your body, especially the hands, and having the virtual 'you' match the feedback from the proprioceptors in your body.

The lack of jitter on the positional feed is also really impressive, is there any potential interference problems with the tech?

Looking forward to more information!

Cheers

-Pete
Last edited by dokworm on Wed May 21, 2014 10:51 pm, edited 1 time in total.

Davideus
One Eyed Hopeful
Posts: 9
Joined: Sat Apr 05, 2014 2:32 am

Re: ControlVR - Full upper body and finger tracking

Post by Davideus »

Wow ! Great to see someone finally making it !! thank you !

IGS Gloves is the best digital gloves around the world... I've seen a lot of them...
So you collaborating with Synertial in official basis?

I see that you are using synertials 12 sensor version of glove, there is also 16 sesor version. Maybe you think of expanding those for great immersive and precise experience of full hand tracking?

Image

amberbluefilms
One Eyed Hopeful
Posts: 1
Joined: Wed May 21, 2014 11:03 pm

Re: ControlVR - Full upper body and finger tracking

Post by amberbluefilms »

Hey Brandon,

I own a film studio in Hawaii called Amber Blue Films and I'll be at USC Cinema this summer. I am very intrigued by this device and your comment about how seeing your body gives you the greatest sense of "self." I am a big consciousness junkie and have studied the human mind on a philosophical level these past few months. I see this as a great step and consider this the discovery of a new "dimension," at which people are now able to experience an entirely new reality. Imagine what will happen when computer graphics will be even greater than human reality, being that we can overclock the amount of particle information from a simulation to give a person a greater omniscient sense of awareness. Anyways, I'll save that discussion for another time.

Speaking of which, I'd definitely enjoy stopping by one day while I'm in LA. I also see that there could be a huge documentary about this technology being made (you guys should look into that). Check out my personal site http://www.cagnation.com if you have the time,

Send me an email at christian@amberbluefilms.com

Cheers

Mystfit
One Eyed Hopeful
Posts: 1
Joined: Tue Aug 06, 2013 8:14 am

Re: ControlVR - Full upper body and finger tracking

Post by Mystfit »

This is brilliant to see! I'm super interested in what you've you're showing here since I've been attempting something similar for my Masters Thesis with teensy microcontroller powered bend sensors and gutted Razer Hydras.

What you've got going here though looks waaay in advance of anything that I've managed to get going. If you're planning on doing a kickstarter then I'd probably sign up first day.

Kevin-Comerford
One Eyed Hopeful
Posts: 2
Joined: Sun May 04, 2014 12:17 am

Re: ControlVR - Full upper body and finger tracking

Post by Kevin-Comerford »

Davideus wrote:Wow ! Great to see someone finally making it !! thank you !

IGS Gloves is the best digital gloves around the world... I've seen a lot of them...
So you collaborating with Synertial in official basis?

I see that you are using synertials 12 sensor version of glove, there is also 16 sesor version. Maybe you think of expanding those for great immersive and precise experience of full hand tracking?
Synertial is a large part of the development of this product. They have been developing motion capture suits for professional applications for years. The VR environment allows their work to expand to a larger market.

The version in the video is actually the 7 sensor version of the IGS glove. We are trying to keep cost down for the consumer version, so the fewer sensors the better. That being said, our interpolation is rather accurate and feels entirely natural. The 12 and 16 sensor gloves will still be available to professional market.

Davideus
One Eyed Hopeful
Posts: 9
Joined: Sat Apr 05, 2014 2:32 am

Re: ControlVR - Full upper body and finger tracking

Post by Davideus »

There is clearly visible difference for example between 7 & 12 sensor versions. It's possible to correct this with software interpolation, forward kinematics? I'm a little skeptical for that..

Image

Can you tell approx consumer price for 12 or 16 sensor if it can be made?

Shelton
One Eyed Hopeful
Posts: 1
Joined: Thu May 22, 2014 9:15 am
Which stereoscopic 3D solution do you primarily use?: Head Mounted Display (HMD)
Location: Houston, TX

Re: ControlVR - Full upper body and finger tracking

Post by Shelton »

Hey Brandon, long-time fan of your work (since around flower warfare days). I'm looking forward to the impact you'll be making on the gaming and video industries in your future.

I work in heavy industry robotics and wanted to tip you off to the phalange load cells NASA JSC has been using in their latest Robonaut rev. Though not necessary for motion tracking, you may want to keep them in mind for any future hopes of tactile input. NASA claims they are the smallest 6-axis load cells available, and they are open for commercial licensing. Check them out if you have a chance.

http://www.nasa.gov/centers/johnson/tec ... sensingCat - (Look for "Phalange Tactile Load Cells" under "Sensing")

I occasionally visit a couple augmented reality companies in your area and would love to stop by your studio to see this.

Andy

User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11394
Joined: Sat Apr 12, 2008 8:18 pm
Which stereoscopic 3D solution do you primarily use?: S-3D desktop monitor

Re: ControlVR - Full upper body and finger tracking

Post by cybereality »

This looks awesome. You have at least one buyer (me!).

desther
One Eyed Hopeful
Posts: 3
Joined: Fri May 23, 2014 5:53 am

Re: ControlVR - Full upper body and finger tracking

Post by desther »

Good luck with kickstarter. You can count me in!

simonion
One Eyed Hopeful
Posts: 6
Joined: Sat Mar 02, 2013 3:05 pm

Re: ControlVR - Full upper body and finger tracking

Post by simonion »

Looks awesome! :woot
How's the latency?
When looking at the video it seems to be some latency to what appears on the screen but how perceptible is it in the rift and do you think you are going to make any headway with it before release?
Also wondering if you thought anything about a control scheme to use with gloves for movement while using these?
I think that is probably the thing that needs the most work right now since ODTs ain't really gonna make it for most people.
Do you think some discrete movements with your fingers could be used(depending on if the gloves are accurate enough to pick it up)?

Can't wait for the kickstarter!

User avatar
V8Griff
Sharp Eyed Eagle!
Posts: 450
Joined: Fri Mar 01, 2013 11:22 am
Which stereoscopic 3D solution do you primarily use?: Head Mounted Display (HMD)
Location: UK

Re: ControlVR - Full upper body and finger tracking

Post by V8Griff »

simonion wrote:Looks awesome! :woot
How's the latency?
When looking at the video it seems to be some latency to what appears on the screen but how perceptible is it in the rift and do you think you are going to make any headway with it before release?
Also wondering if you thought anything about a control scheme to use with gloves for movement while using these?
I think that is probably the thing that needs the most work right now since ODTs ain't really gonna make it for most people.
Do you think some discrete movements with your fingers could be used(depending on if the gloves are accurate enough to pick it up)?

Can't wait for the kickstarter!
The video should look awesome those gloves are £16k each.

Hopefully the Kickstarter project will have the same accuracy and low latency for the low price being projected.

Kevin-Comerford
One Eyed Hopeful
Posts: 2
Joined: Sun May 04, 2014 12:17 am

Re: ControlVR - Full upper body and finger tracking

Post by Kevin-Comerford »

simonion wrote:Looks awesome! :woot
How's the latency?
When looking at the video it seems to be some latency to what appears on the screen but how perceptible is it in the rift and do you think you are going to make any headway with it before release?
Also wondering if you thought anything about a control scheme to use with gloves for movement while using these?
I think that is probably the thing that needs the most work right now since ODTs ain't really gonna make it for most people.
Do you think some discrete movements with your fingers could be used(depending on if the gloves are accurate enough to pick it up)?

Can't wait for the kickstarter!
The latency is very minimal and barely perceivable. There is also absolutely no optimizing being done in this video. The consumer version will be even better than what's shown.

We are already looking into the ideal control scheme. The demo shown already works with an xbox controller. You can actually see the controller sitting on my desk at the beginning, but the movement didn't make it into the video. The system is already good enough to pick up discrete movements such as a mouse click. You could potentially use small motions as inputs, it's all up to the developer. We already have a rock paper scissors game working.

User avatar
WhoIsRich
One Eyed Hopeful
Posts: 43
Joined: Sun Feb 24, 2013 7:32 am
Which stereoscopic 3D solution do you primarily use?: S-3D HDTV
Location: UK

Re: ControlVR - Full upper body and finger tracking

Post by WhoIsRich »

This looks great!

Looks more practical than lots of flex sensors, like in Imogen Heap's recent glove while avoiding the occlusion problems that you get with camera tracked gloves.

Now it's just a question of pricing, I hope it's closer to the cost of a DK2 than 16k :|

desther
One Eyed Hopeful
Posts: 3
Joined: Fri May 23, 2014 5:53 am

Re: ControlVR - Full upper body and finger tracking

Post by desther »

WhoIsRich wrote: Now it's just a question of pricing, I hope it's closer to the cost of a DK2 than 16k :|
Yea, this has to be the lowest price possible, even by cost of lower tracking precision (lower amount of sensors etc) if the creator hope this to be supported by software developers and bought by mainstream users.

I think the sweet spot here is around 300$ for "lite" version (more options is a MUST here, just like with PrioVR). Of course the lower is price, the better.

Oculus Rift wouldnt be so heavily supported if it costs 2000$+ (and to be fair, even that would be multiple times lower price than competition back then).

Zoide
Binocular Vision CONFIRMED!
Posts: 345
Joined: Fri Nov 18, 2011 1:32 am

Re: ControlVR - Full upper body and finger tracking

Post by Zoide »

I agree, price is crucial. There's already a high-end market for multi-thousand dollar gloves (movies, military, medical, research, etc.), but the real breakthrough would be expanding into the mass market.

zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: ControlVR - Full upper body and finger tracking

Post by zalo »

https://twitter.com/BrandonJLa/status/4 ... 2696361984

So under $1000... rather expensive.

Aiming for $350 is admirable. Just still not within the realm of fiscal feasibility.

User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11394
Joined: Sat Apr 12, 2008 8:18 pm
Which stereoscopic 3D solution do you primarily use?: S-3D desktop monitor

Re: ControlVR - Full upper body and finger tracking

Post by cybereality »

Honestly, if he solved the glove issue, I'd be willing to pay a decent price.

User avatar
Deop
One Eyed Hopeful
Posts: 22
Joined: Wed Apr 17, 2013 2:42 am
Which stereoscopic 3D solution do you primarily use?: Head Mounted Display (HMD)

Re: ControlVR - Full upper body and finger tracking

Post by Deop »

Looking forward to the kickstarter.
Price-wise I think anything beyond $150 will have a hard time reaching a large audience.

rollerphreak
One Eyed Hopeful
Posts: 1
Joined: Sun May 25, 2014 5:45 am

Re: ControlVR - Full upper body and finger tracking

Post by rollerphreak »

I think this kickstarter project will skyrocket, since many are looking for this. Mainstream is the key here, since the more people get to "play" with it the more supporter it will receive.
Don't know but I feel we are leaping quite closer to the Matrix effect...

Zoide
Binocular Vision CONFIRMED!
Posts: 345
Joined: Fri Nov 18, 2011 1:32 am

Re: ControlVR - Full upper body and finger tracking

Post by Zoide »

Just one week until the E3 demo next Tuesday! :)

User avatar
V8Griff
Sharp Eyed Eagle!
Posts: 450
Joined: Fri Mar 01, 2013 11:22 am
Which stereoscopic 3D solution do you primarily use?: Head Mounted Display (HMD)
Location: UK

Re: ControlVR - Full upper body and finger tracking

Post by V8Griff »

Kickstarter here:

https://www.kickstarter.com/projects/co ... on-and-mor

Impressive technology but surely something like this should work with other technologies like PrioVR for example , otherwise we're going to end up with so many peripherals but no cohesion?

User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11394
Joined: Sat Apr 12, 2008 8:18 pm
Which stereoscopic 3D solution do you primarily use?: S-3D desktop monitor

Re: ControlVR - Full upper body and finger tracking

Post by cybereality »

I backed it for $600. I hope they make it, gloves are a missing piece to the puzzle.

User avatar
mindless
Cross Eyed!
Posts: 100
Joined: Thu Dec 06, 2012 11:19 am

Re: ControlVR - Full upper body and finger tracking

Post by mindless »

Would love to jump in on this, but unfortunatly i'm now totaly skint. gonna have to sit this one out, but it looks really promissing, hope they do well.

GrantHnS
One Eyed Hopeful
Posts: 1
Joined: Tue Jun 10, 2014 9:02 am

Re: ControlVR - Full upper body and finger tracking

Post by GrantHnS »

Definitely the most impressive finger, hand, and arm tracking I've seen yet! Such and exciting time...

Definitely in to support you guys! Good luck on the KickStarter!

desther
One Eyed Hopeful
Posts: 3
Joined: Fri May 23, 2014 5:53 am

Re: ControlVR - Full upper body and finger tracking

Post by desther »

This looks very nice. Good job!

metronome
Cross Eyed!
Posts: 114
Joined: Sat Mar 16, 2013 12:42 pm

Re: ControlVR - Full upper body and finger tracking

Post by metronome »

I soooo want one of these they look amazing... $300 a bit steep for me atm... btw are there any plans for a more consumer / lower price version in the future? Just im wary that all these oculous vr type projects are still targetted for development studios....

Post Reply

Return to “Physical VR Tools”