6 DOF Head Tracking Ideas

[divide]

I think I'll start by compiling the latest revision of the original PTAM, since it has a win32 project bundled in, and then try to integrate the optimization by the ROS team (AGAST library, limited map size, auto-initialization).
The system how I see it so far would be PS3 Eye+PTAM to have absolute 6DOF reference with 20ms-30ms latency, and gyros+accel would be used to go down to 8ms latency AND match PTAM tracking with real word dimension (accel when moving), get the horizon (accel when static, gravity vector) and still have orientation working when the tracking is temporarily lost (gyros).
BTW anyone know what gyros/accel are embedded within the Rift ? (brand/product id)

[FingerFlinger]

@divide I think I want to try a subset of that. AGAST library, and try to optimize for 2D translation. I still need to finish reading (and digesting) the research papers, but my PS3 Eye is supposed to arrive tomorrow, so I want to at least get it working with the reference implementation by the end of the weekend.

EDIT: http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15312

I started a new topic specifically for discussing the implementation of PTAM and related techniques. I'll be posting my progress there, if I have any. I think it would be good to have a more focused discussion as we inevitably work through library/dependency issues and specific implementations.

[brantlew]

For a quick and dirty off-the-shelf method, it seems like sticking a Hydra on your head is currently the easiest option.

Not sure about that, Carmack has been experimenting with the Hydra ( http://www.youtube.com/watch?v=wt-iVFxgFWk , starting 2h05) and he found out the 3d positioning is not linear; he tried building a calibration map for it but it ended up being too complex.

True. But there is a lot of anecdotal evidence starting to come in around this forum that the Hydra works good enough, and can even be worn. Wearing the device would not allow you to accurately perform global maneuvers like laying down on the floor, but you could still achieve the parallax and leaning effects using just relative motions (ie. head position relative to your belt). You might even be able to achieve some global motions if you attached one sensor to your shoe and one to your head. Then you would have a pretty decent "squat" measurement. So other than the technical details of mounting the device on your body and battery powering it, I think it has real potential as a hobbyist solution. I'll be following the Hydra threads closely for the next few months.

[cybereality]

From my experiments the Hydra is pretty accurate within a 16" radius from the base station. As you move further way it seems to have a bias toward a particular angle. So if you kept the wand facing straight forward, as you translated the position away from the base it would start to veer off at an angle. It is not so extreme to make the device unusable, it is still very good all things considered. But its not perfect.

[Mitbekommer]

Regarding android as headtracker.

I tried this once.

Problems:

1. Where to mount the android on the head?
2. How to interpret data to use it in a game?

[brantlew]

I've played with phones as head-trackers before. One simple thing you can do is buy a phone belt clip. Then wear a baseball cap and hook the phone clip to the back strap of the cap and track the back of your head. A great on-the-go solution.

[Mitbekommer]

I've played with phones as head-trackers before. One simple thing you can do is buy a phone belt clip. Then wear a baseball cap and hook the phone clip to the back strap of the cap and track the back of your head. A great on-the-go solution.

So you haven't experienced that rotating the device around 360° does not give you values within 360°? I found that the results are not reproduceable at the same angles.

[brantlew]

It depends on the phone model and the API version. The iPhone for instance can tap into GPS Location services as well as its magnetometer compass to give you absolute "heading" values relative to magnetic or true north. For a price though - those values don't update as frequently, but you can use them to calibrate the raw gyroscopic angles. If you are using third-party apps they may not be passing those values through and you will have to code your own app to get directly at the API.

http://developer.apple.com/library/ios/ ... renceFrame

[Mitbekommer]

I don't use an iphone, fortunately, but an android phone. Using it, you can go into debug mode and forward the values you want in real-time to your computer using your own java-coded app. So getting the values is not hard. But as I said: Put the device in the same position twice and you get different values. Might be due to the computer's magnetic field.

GPS is useless when being used within these small movements and angles.

[brantlew]

Well I have nothing but bad things to say about the iPhone in terms of development, but it's corrected angles are pretty good. I just looked at the sample link you provided. That example only uses the accelerometers for tracking rotation which is going to provide a pretty bad approximation. You would do much better with the values returned by the gyroscopes. And even better if your phone supports magnetometer corrected angles. Android also has API's for getting these sensor fused values.

Caveat: A magnetically or electrically noisy environment can cause the corrected values to drift but at a very manageable pace.

Also, GPS is relevant for defining angles in reference to true north but otherwise it's not much use.

[Mitbekommer]

I replaced the example with sensor fusion some time ago. Now the compass, accelerometer and some other sensor are being fused together. But the results did not improve. My very phone has no gyroscopes. True north is unimportant. Also inside your living room you don't have gps reception.

Question is, which hardware to use.

[brantlew]

Just drawing attention to a related thread that I started about magnetic tracking.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15431

[RED]

http://www.vectornav.com/products/vn100-smt?id=36 Aside from the price point being out of reach for a gaming HMD, can anyone tell me why a device using sensor fusion like this would or wouldn't be a great solution?

Edit: I believe the Adjacent Reality tracker currently in use is similar to (and outperforms) this tracker and thus I've answered my own question?

[brantlew]

Here is a great video about the ins and outs of inertial sensors. Around the 23 minute mark they talk a lot about cumulative integration error and how that is problematic for position tracking.

[youtube-hd]http://www.youtube.com/watch?v=C7JQ7Rpwn2k[/youtube-hd]

I've personally gone through the exercise of filtering and integrating raw inertial signals and can attest to how problematic it is. Even just for velocity calculations (single integration) you end up not just with the wrong velocity - but a non-constant velocity.

Here's an example of moving an inertial sensor at a constant velocity for about 1 meter. The blue line is the acceleration signal and the green line is the single integration velocity. You should expect the green line to move up to a near constant level and stay there until the motion stops. Instead the error bias in the signal (which can be caused by small tilts in the orientation and a gravity bias or just plain sensor fluctuations) causes the velocity to decrease until it shows the sensor moving backwards!!!

raw_imu.jpg

And smoothing ("dampening") the signal does nothing to avoid this. Here's the same data smoothed with a 100ms mean filter and integrated. It just makes the velocity changes smoother but the shape of the curves are the same. My sensor still looks like it started moving forward and then went backwards.

smoothed_imu.jpg

Dealing with inertial sensors is just infuriating. It's sort of like trying to stop a marble on a tiny dot in the center of a perfectly flat disk. Any small overshoot is amplified by the continually increasing speed of the marble so that tiny errors result in huge swings of the orbit. That's the way that I imagine even a carefully constrained inertial head model working out. Just like that marble - swinging wildly back and forth and not able to converge to a solution.

[RED]

Leap Motion (or a couple of them) attached to an adjustable microphone stand facing the HMD-wearing player's head? Does anyone know enough about how the Leap works to verify if this could be a future solution to translational tracking? Just a thought.

[brantlew]

The demos I have seen show the Leap SDK primarily tracking "pointy" objects like fingertips and pens. I don't know how generic the interface and technology is for tracking other types of objects. Also the SDK may need to have built in knowledge of a skeletal system to track hands properly. It is unlikely that a head/torso skeletal model would be built into the SDK.

[Naru]

I think the main ability of the leap is creating a point cloud of what it sees. The software itself handles the finger, hand, and object recognition. If someone were to create an app for detecting head motions using the point cloud data then it could be possible. I find two leaps to be an expensive solution though. I'm sure a much neater and less wasteful solution to this problem will come up in the next several months.

[mahler]

Something I thought about a while back, but forgot to ask here.

Regarding drift for both orientation and position:
How much could the drift theoretically be reduced by adding more than one IMU to a body and taking average values?

[EdZ]

Something I thought about a while back, but forgot to ask here.

Regarding drift for both orientation and position:
How much could the drift theoretically be reduced by adding more than one IMU to a body and taking average values?

Very little, if at all. The drift comes from integrating sensor noise, so you'd pretty much get the same drift from integrating the average of the two noises.

[FingerFlinger]

Just to expand on Edz's point, to effectively fuse multiple sensors, the sensor error should be independent of each other, meaning that they measure by different mechanisms. Hence, acc, gyro and mag, all to measure orientation.

[PatimPatam]

Hello there! Well, after a few months of work, here's my personal take on the subject!

http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=16072