Rift full-body avatar with Kinect, Razer Hydra, and PS Move

Post Reply
User avatar
TTakala
One Eyed Hopeful
Posts: 37
Joined: Wed Oct 17, 2012 10:41 am
Location: Finland
Contact:

Rift full-body avatar with Kinect, Razer Hydra, and PS Move

Post by TTakala »

Over one month ago we released the downloadable TurboTuscany demo with support for Oculus Rift with Kinect, Razer Hydra, or PS Move. Since then we released a new video demonstrating full-body avatar control using Kinect + PS Move combination:
http://www.youtube.com/watch?v=-rIL-g-Av4w

Below I sum what we learned while developing the TurboTuscany demo. Some of our findings are consequential, while some are common knowledge if you have developed stuff for Razer Hydra, Kinect, or PS Move before.

Latencies of used devices:
Oculus Rift > Razer Hydra > PS Move > Kinect

Body tracking with Kinect has an easily noticeable lag, has plenty of jitter, and the tracking fails often. Nevertheless, Kinect adds a lot to the immersion and is fun to play around with.

From all the positional head tracking methods available in our TurboTuscany demo, PS Move is the best compromise: big tracking volume (almost as big as Kinect's) and accurate
tracking (not as accurate as Razer Hydra though). Therefore the best experience of our demo is achieved with Oculus Rift + Kinect + PS Move. Occlusion of the Move controller from PS Eye's view is a problem though for positional tracking (not for rotational).

Second best head tracking is achieved with combination of Oculus Rift, Kinect, and Razer Hydra. This comes with the added cumbersomeness of having to wear Hydra around the waist.

My personal opinion is that VR systems with a virtual body should track the user head, hands, and forward direction (chest/waist) separately. This is so that the user can look into different direction than the direction where he is pointing a hand-held tool/weapon, while walking in a third direction. In TurboTuscany demo we achieve this with the combination of Oculus Rift, Kinect, and Hydra/Move.

Latency requirements for positional head tracking

The relatively low latency of Razer Hydra's position tracking should be low enough for many HMD use cases. If you're viewing objects close, the Hydra's latency becomes apparent however when moving your head. Unless STEM has some new optimization tricks, it will most likely have different latency (higher?) than Hydra because it's wireless.

If head position tracking latency is less or equal to Oculus Rift's rotational tracking, that should be good enough for most HMD applications. Since this is not a scientific paper that I'm writing here, I won't cite earlier research that suggests latency requirements in milliseconds.

Because we had positional head tracking set up to track the point between eyes, we first set Oculus Rift's "Eye Center Position" to (0,0,0) which determines a small translation that follows the orientation of Rift. But we found out that the latency of our positional head tracking was apparent when moving the head close (>0.5 meters) to objects, even with Razer Hydra. Therefore we ended up setting "Eye Center Position" to the default (0, 0.15, 0.09), and viewing close objects while moving became much more natural. Thus, our positional head tracking has a "virtual" component that follows the Rift's orientation.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by cybereality »

Pretty cool, except the video thumbnail looks like you're gonna kill grandma.
User avatar
TTakala
One Eyed Hopeful
Posts: 37
Joined: Wed Oct 17, 2012 10:41 am
Location: Finland
Contact:

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by TTakala »

cybereality wrote:Pretty cool, except the video thumbnail looks like you're gonna kill grandma.
It's not that sinister, watch the video ;)
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by cybereality »

I did watch it, it's looking really good.
User avatar
TheHolyChicken
Diamond Eyed Freakazoid!
Posts: 733
Joined: Thu Oct 18, 2012 3:34 am
Location: Brighton, UK
Contact:

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by TheHolyChicken »

Looking great, TTakala. How proficient did you find the Kinect sensor for tracking the orientation of the player's hips? Was it able to give you a reasonably accurate orientation? What about when the player is facing away from the camera? I'd like to attempt to determine the difference between the head and body orientation using a combination of the Rift & Kinect, but don't know if I'm wasting my time!
Sometimes I sits and thinks, and sometimes I just sits.
User avatar
TTakala
One Eyed Hopeful
Posts: 37
Joined: Wed Oct 17, 2012 10:41 am
Location: Finland
Contact:

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by TTakala »

cybereality wrote:I did watch it, it's looking really good.
Thanks cyber!
TheHolyChicken wrote:Looking great, TTakala. How proficient did you find the Kinect sensor for tracking the orientation of the player's hips? Was it able to give you a reasonably accurate orientation? What about when the player is facing away from the camera? I'd like to attempt to determine the difference between the head and body orientation using a combination of the Rift & Kinect, but don't know if I'm wasting my time!
Try our demo out with Kinect and a wireless game controller to see for yourself. Moving forward/backward with the controller reveals what OpenNI thinks the torso orientation to be.

Kinect detected torso orientation is good enough for our purpose of tracking the forward direction. It does fail relatively often however, especially if you are facing away from Kinect. Bending knees and elbows can help the OpenNI tracking a little though. The torso orientation angle has jitter of 5-20 degrees, and we use a simple Kalman filter to smooth it out.

In OpenNI the head rotation is same as body rotation except for whatever reason it fails more often. In our demo we are actually using the head (Rift) and body orientation (Kinect) difference to calculate a rough approximation of Rift's yaw drift. Drift correction based on that method can be enabled by Kinect users.
User avatar
TheHolyChicken
Diamond Eyed Freakazoid!
Posts: 733
Joined: Thu Oct 18, 2012 3:34 am
Location: Brighton, UK
Contact:

Re: Rift full-body avatar with Kinect, Razer Hydra, and PS M

Post by TheHolyChicken »

I'll definitely investigate further, that sounds promising. Thanks, TTakala.
Sometimes I sits and thinks, and sometimes I just sits.
Post Reply

Return to “Oculus VR”