I have been talking with a friend about this topic and we had a crazy idea for a DIY 6DOF head tracker. We probably missed some important point, but I wanted to know if somebody has already tried a similar approach...

We know that a accelerometer is unable to calculate the "forward" vector when one of the axis (X, Y or Z) is pointing directly downwards, so most commercial 6DOF devices are built using a 3-axis accelerometer and a 3-axis gyroscope and the orientation of the sensor is calculated using the information of both sensors. The main problem of this approach is that gyroscopes have some drift and they can become pretty expensive for a DIY project. We have been talking about those issues and I've realized that if we put two 3-axis accelerometers together in a non-orthogonal angle, they would never have their axis pointing downwards at the same time, so we could use the information of both sensors to calculate the direction of the "forward" vector. In addition, accelerometers are much cheaper than gyroscopes so we could have a very cheap 6DOF tracker if this approach works.


Have anybody tried to build a similar device?
Does anybody know if we have missed some technical detail that could make this idea fail?

Basically a triangular grid of RGB points, with white points within each triangle used to identify them. I think the best bet it to try and identify them almost globally; that is there are enough unique identifiers to mean you don't need to repeat them near each other.

@nrp: you are right, I would continue using only the gravity force, but I would avoid the situation described above because the accelerometers wouldn't be placed in an orthogonal angle, so they will never point directly downwards at the same time. However, I don't know if that would be good enough for building a reliable 6DOF tracker...

I haven't talked about including a magnetosphere because that would be a 9DOF tracker (or at least, that is the name that is being used in certain online stores) but I think it should be possible to use a magnetosphere (in addition to the accelerometers) if a greater accuracy is required. My post was about using "2 accelerometers vs 1 accelerometer + 1 giroscope", but we can also talk about "2 accelerometers + 1 magnetosphere vs 1 accelerometer + 1 giroscope + 1 magnetosphere" if you want, it could be an interesting discussion...

It doesn't actually matter which way the accelerometers are pointing. The only measurement they are making is of acceleration, and for the purposes of determining orientation, that is the constant acceleration of gravity. Regardless of how many accelerometers you have and which axes they are or aren't aligned to, the values you're reading out will only let you determine which way gravity is, which means you will not have a yaw measurement, a measurement of the rotation around the gravity vector. Adding a gyro means you do have a yaw measurement, but because it is only relative rotation that is being measured, the measurement isn't anchored to anything physical and it can drift.

Additionally, digital gyros generally sample at a much higher rate than accelerometers, and are also not thrown off by non-rotation movement. A MEMS based motion tracking device needs an accelerometer and gyro at minimum (unless you don't care about anchoring roll/pitch), and preferably also a magnetometer (to get rid of yaw drift as well).

I've played around with a number of different options for tracking motion, and I thought I'd quickly share some of my musings.

My intent is to set up a vr system where:

* you can rotate freely with yaw (you dont have to always be facing a camera or something)

* (relative to a fixed point) acquire body position/offset in 3d

* accurately track relative hand movements (rotation and 3d position)

* accurately track head rotation

I've succeeded with most of this now. Though I haven't put it all together, and I am not yet satisfied with the latency.

The Kinect is nice in that it gives you fairly accurate skeletal pose information across the body (and the libraries make it really easy too). But, it does not do this fast. Of course fast is relative, but with the intent of getting things down to around 20ms or less, the Kinect is nowhere near practical/helpful.

I have used a high speed camera (PS3 eye) and custom OpenCV stuff. I've played with tracking fiducials. I toyed with wiimotes (and with the motion plus) and playstation move controllers. The software to get the moves talking and calibrated was a pain to get going, and unfortunately in its current state adds a bunch of latency. I'm convinced that ultimately you want an AHRS (with 9dom - 3 axis accelerometer, 3 axis gyro, 3 axis magnetometer). There is some work going on to directly reprogram the moves. I imagine that the moves might provide the cheapest avenue for others to explore AHRS.

I worry that overall, bluetooth might end up costing too much in latency.

I think, without necessitating a fixed camera or magnetic tracking, you would probably be able to integrate from a set of AHRSs to find displacement and trigger for things like jumping and crouching without losing much.

---

Using a bunch of ps moves, I set up a fun system. I made a number of straps that would hold a move each. The moves were calibrated. I'd strap one to my forearm, one to my upper arm, one to my chest, and one to my head (next to an hmd). I also held a ps navigation controller.

I did some further "baselining" in code. The user would go into a few different poses and I would calibrate. I constructed a skeleton, very similar to how I imagine Carmack is. It took into consideration the offset of the position of the strap/device from the actual point of rotation. Because I had the "body" sensor I could get an accurate 3d position of the head, upper arm, lower arm, and hand relative to the body basis through the composition of the rotations.

I downloaded the source sdk and started tweaking half life. I didn't get too far with it overall, but I did incorporate the pose tracking for the body and head. At some point, I'd like to return and incorporate the arm/hand tracking to be able to aim and shoot realistically.

It was really fun and immersive to put on an hmd, view things in 3d (thanks to tridef), walk around in game moving my head and body convincingly. I even would physically lay on my side and look up under things. For example, at the beginning of HL2 you arrive on a train. One of the first things I did was lean my body outside of the train door and move my head around. I also went on my side and looked up under the seats of the train. I got out of the train and looked up under the steps getting off the train. I'm sure I looked really stupid in meatspace, but it was awesome.

(I showed it to a few others, and they quickly pointed out problems with it. Why were the bullets shooting out from my face? Why could you lean through walls?)

---

How to make it better:

I don't think I'm doing the calibration quite right or something as I end up with a bit of drift.

I think if I added a fast camera mounted above the user looking directly down, and had some ir leds on top of the head, I could have a satisfying, cheap, convincing, and fast 3d position/offset. This would still allow me to rotate about freely in the ways that I care about. I could use this to get an absolute positioning of the head and use that as the reference basis instead of the body. This is likely best because the head tracking is most important and noticeable. Going the other way led to jittery head movement. This would also let me handle jumping and crouching convincingly.

Instead of each ps move sending out their signals via bluetooth, getting them in software, and processing the rotation calcs, I should reprogram them to do the calculations onboard and only output the final rotation quaternion, wire them together and probably just send the full output through usb.

---

I'm really excited about getting ahold of the rift and putting this all together!

@cms: That sounds like a very cool project. Congrats on getting it past just the initial movement tests and actually customizing to a game environment. There is a thread dedicated to just this sort of thing here.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=15040

You should post some videos of the system in action.

This is very cool CMS. My wife and I recently started a project similar to this with a few differences. We are looking at using Wii motion plus controllers for tracking hand movement however I am starting to have doubts about the accuracy of the Wii motion plus, it is also a rather difficult piece of hardware to work with on windows as all the libraries people have written haven't been touched in years and have known issues. How did you find using the PsMove? what bluetooth stack would you recommend for working with Move on PC?

Have you seen new Leap 3D demo? It looks really responsive.
http://www.theverge.com/2012/6/26/31185 ... e-controls

This looks like a nice chip? I guess all we need is a small interfaceboard with this chip on it and we would have a very nice 'headtracker' ?
http://invensense.com/mems/gyro/mpu9150.html

The specs looks great.

With unlimited resources you can't beat high-resolution multi-camera setups.

@Krenzo: So just out of curiosity, what are your experiences with RF? I guess at a small enough wavelength you could (in theory) differentiate some really small motions at a very long range. There must be a bunch of practical issues with it though. Would you use something like a phased array transmitters and a bunch of antennas placed on the body - something like a mini-GPS? Or do you try to bounce signals off of metallic markers or something? I guess it would be subject to all the same noise and multi-path problems that plague all radio. What are the practical limits of precision for it?

Cool - looking forward to it. One last question. Assuming you could transmit with as much power as you wanted, what are the practical range limits of the technology. Could you detect motion < 1cm at 100m ? Is transmission at that distance practical and if not are the limits technological, cost, or just regulatory?