Head Tracker Roundup

Post by **brantlew** » Thu Mar 15, 2012 4:44 pm

With the ST1080 coming up, I'm in search of head tracking solutions. I know this has been covered in a lot of threads, but I figured why not try to distill all the information into one place. Maybe at some point this could become a reference article, but for now I just want to accurately gather all the information. I'm going to list what little I know at the top and fill in the details as I learn more and people chime in with additional information.

Fixed Position / Limited Angle Trackers

These are basically all the optical trackers that require you to be facing a sensor and allow a limited angular rotation of the head. They are highly accurate, have no calibration drift, have very little lag, but have a limited angular range. Useful for cockpit simulators.

1. TrackIR - Optical camera tracker with a proprietary API and a fair amount of native game support. Users must wear and calibrate 3 LEDs on their head. Supported games implement absolute (and highly stable) camera positioning.

2. FreeTrack - Same as TrackIR except open source.

3. Wiimote IR Camera - The Wiimote controller can be mounted facing the player with the user wearing IR LEDs - similar to FreeTrack.

4. Playstation Move + Eye - The PlayStation Move controller can be mounted to the head and used as a head tracker. The API requires that the PlayStation console must be used as an interface bridge.

Fixed Position / 360 Degree Trackers

Trackers that allow full rotation but require the user to stay within a very limited space.

1. Razer Hydra - The Hydra device emits a magnetic field that can be detected by two controllers. Very accurate 6D positioning. (Are there calibration drift issues?). The controllers can be mounted to the head to implement head tracking but the device has a short range and is wired so free movement is limited. (What is the interface method?)

Unrestricted Motion Trackers

Trackers that allow for 360 rotation and unlimited (or very wide) range of movement. IMU devices fall into this category using both inertial/gravitational/and sometimes magnetic sensors to detect orientation and motion in space. Unfortunately these devices suffer from calibration drift and may need to be recalibrated quite often. Devices that only make use of inertial sensors are more susceptible to drift. Adding magnetic sensors generally improves the stability of the device but they are susceptible to environmental electro-magnetic fluxuations and can also drift.

1. Wiimote (Plus) - Inertial tracking only. Simple and well supported interface.

2. Playstation Move - 9DOF (inertial + magnetic). Difficult to interface. The open source API's for gathering data from this unit are incomplete. (Does the Sony API via PlayStation allow access?)

3. Sparkfun Razor - A high quality integrated 9DOF sensor module. Not end-user packaged so the user must construct a housing and interface. Immature open source code for retrieving and computing orientation from the sensors.

4. iPhone/iPod Touch - These products have a good quality IMU+magnetic sensor package. The data can be sent to a PC using the Sensor Data App over WiFi. Unfortunately the sensor data stream over WiFi has a periodic stutter which makes the iPhone a bit unstable as a head tracker.

5. CKDevices Mongoose - 9D0F IMU module that is board compatible with the Sparkfun Razor. Additionally includes a barometric pressure "altimeter" sensor.

6. Hillcrest FreeSpace - Packaged inertial tracking unit with proprietary API support for reading orientation values. The reference kits include USB or Bluetooth and are preloaded with sensor fusion firmware and Windows drivers. With the libfreespace interface library, they are a near plug-and-play solution. (This is the tracker used by the Oculus Rift prototype)

I know this is tread-bare, but I hope to flesh it out soon. Input welcome. Please tell me what I left off the list and what I got wrong.

Post by **WiredEarp** » Thu Mar 15, 2012 5:18 pm

Nice idea Brantlew.

Re the Hydra:

Range is about 1.5-2m, so is fine for a head tracker, even for stand up VR. There is no drift, but having metal objects or magnetic sources nearby can cause interference.
The interface method is currently only mouse emulation. The Hydra can be taken apart to lighten the weight and make it smaller. If you do this, be VERY careful of the micro ribbon type cable inside. Its one of those crap paper thin ones that break extremely easily - I've broken mine and need to suss out a replacement somehow (which is why I haven't written a Hydra -> TrackIR interface yet). Its so small there is no chance of soldering the 6 wires manually, unless you are some micro solder guru.

Post by **cybereality** » Thu Mar 15, 2012 6:11 pm

I have nothing to add, but I support this thread.

Post by **FingerFlinger** » Thu Mar 15, 2012 6:45 pm

Does anybody have first-hand experience with the Hillcrest units? I didn't realize that the reference packages were so cheap.

Post by **FingerFlinger** » Thu Mar 15, 2012 6:57 pm

Also, here's another IMU that looks interesting. http://store.ckdevices.com/products/Mon ... nsor-.html

Post by **brantlew** » Thu Mar 15, 2012 7:35 pm

One that I nearly added to the list except that it's not really general purpose is the Vuzix 6TC. This IMU plugs into the Vuzix Wrap units for integrated head tracking. I use it a lot, and I'm really going to miss it because it's so convenient and so well implemented. Just plug it into your USB port and grab the orientation coordinates using a simple API. I wish there was a way to hack it so that it could be used independently from the Wrap.

That's really the thing that is missing from this list that everyone badly needs. A packaged, stand-alone 9DOF IMU with a well done and simple interface for retrieving orientation values. The FreeSpace reference kits come very close to achieving this but it seems that they only include inertial sensors.
http://www.hillcrestlabs.com/products/f ... srk-bt.php

If they had a 9DOF IMU kit then FreeSpace would be a great turn-key solution. Has anybody found anything like that so far?

Post by **CyberVillain** » Fri Mar 16, 2012 1:56 am

FingerFlinger wrote:Also, here's another IMU that looks interesting. http://store.ckdevices.com/products/Mon ... nsor-.html

It has the same internal sensors as the Sparkfun and is AHRS compatible (It will work with FreePIE), so I think Sparkfun IMU and this one is interchangeable. Sparkfun is another 10 dollars...

Post by **brantlew** » Fri Mar 16, 2012 7:31 am

@FingerFlinger: Nice find. I guess the barometric pressure is for absolute altitude. From what I gather it's only accurate to about 20 meters so not useful for character tracking, but good for model aircraft hobbyists.

@CyberVillain: So this is a drop-in replacement for the Sparkfun, right? Have you played with that device much in the last couple of months? It seems you were disappointed with the drift. What are your impressions of its performance and ease of use?

Post by **CyberVillain** » Fri Mar 16, 2012 7:40 am

brantlew wrote: @CyberVillain: So this is a drop-in replacement for the Sparkfun, right? Have you played with that device much in the last couple of months? It seems you were disappointed with the drift. What are your impressions of its performance and ease of use?

It was very responsive and easy to use except for the drift (Which could be worse on mine unit then the average). And now with FreePIE its ease of use is very high

I should get an RMA for my unit, but the Swedish reseller I bought from was a pain in the arse to deal with so I havent gotten to it.

edit: Yes its a drop in replacement, it has a Arduino bootloader so you can load both AHRS software or FreeIMU software in to it, both supported in FreePIE and they do give a little different experience (Different use of internal 9DOF sensors to output 3DOF)

Post by **brantlew** » Fri Mar 16, 2012 8:04 am

@CyberVillain: How would you characterize the drift? Could you give an estimate of how many degrees off in yaw and pitch you drift per minute? (or longer time span if that is easier)

Post by **CyberVillain** » Fri Mar 16, 2012 8:14 am

brantlew wrote:@CyberVillain: How would you characterize the drift? Could you give an estimate of how many degrees off in yaw and pitch you drift per minute? (or longer time span if that is easier)

Heres a clip i did, no filtering done in freepie so its a bit jumpy

[youtube]http://www.youtube.com/watch?v=L4plqHQpewY[/youtube]

Post by **brantlew** » Fri Mar 16, 2012 8:47 am

Hmmm. It does look very noisy, but I didn't notice much of an overall trend over that short time frame. It seems to average out to zero. Maybe you do have some faulty sensors...or a very electro-noisy environment?

Thanks for the info.

Post by **CyberVillain** » Fri Mar 16, 2012 8:51 am

brantlew wrote:Hmmm. It does look very noisy, but I didn't notice much of an overall trend over that short time frame. It seems to average out to zero. Maybe you do have some faulty sensors...or a very electro-noisy environment?

Thanks for the info.

Well one trend you can see is that after you move your "head" it drifts quite alot and then stabilizes. I do not think I have more background noise than the average person

Post by **VRgamesterz** » Fri Mar 16, 2012 10:13 pm

I have the Hillcrest USB model, Im using it right now in the Trimersion Gun for tracking because the Trimersion tracker was horrible. There is little drift at times but pretty much I love it for the price and job that it does. Very small like the size of a quarter and you can turn it sideways as I did to fit in the gun and still worked well! I haven't done to much testing besides just FPS aiming.

I will try more testing sometime later on after I get this projector finished.

Ku

Post by **Hooves** » Fri Mar 16, 2012 10:56 pm

you have my idea with the projector. Which one did you go with? My original idea was to use the Hydra for tracking but didnt know if there was an interface I could use to have it function the way I want it.

Man if there was a comapany out there that realised how ready this market is to burst, there could be a killing made!

Do any one you fellas have a teamspeak server we can jump on to share Ideas or get questions answered in real time? If Not I have one we can go to.

let me know!

Post by **brantlew** » Fri Mar 16, 2012 11:22 pm

@VRgamesterz: Oh cool. I didn't see that little guy until you mentioned it. Nice that it's already got the USB interface integrated, so the unit is pretty much ready to go with their lib. And that's the best part about these Hillcrest units is the fact that they have a mature driver instead of a bunch of cobbled together open-source. It's also so small you could probably just foam tape it to an HMD without much fuss. The underside of the HMZ-T1 would probably be a convenient place. I just wish these guys would do a 9DOF module. How often do you need to recenter because of drift with that device?

Hey, can you also give me a brief "flavor" of what their software API is like. Is it just your basic C calls into a dll, or maybe on OCX? Can they return computed values like yaw, pitch, and x, y? Can they also return raw sensor data?

Post by **cybereality** » Sat Mar 17, 2012 5:30 pm

Hmm, the jitter in that video looks horrible. Nowhere near acceptable quality. However the end part looked OK, so maybe its just a bug with mouse emulation.

Post by **brantlew** » Sat Mar 17, 2012 8:04 pm

I think that was the raw signal without any low-pass filtering, so I'm not surprised by that type of base noise level. The bigger problem was how it seemed to swing to a completely new heading every 2 or 3 seconds. The other videos I have seen of this device haven't looked like that at all, so you gotta assume it's either a bum sensor - or else some kind of software glitch.

Post by **PalmerTech** » Sat Mar 17, 2012 9:22 pm

The Hillcrest tracker has a lot of drift, in my experience. Not at all acceptable for head tracking.

Something to throw out there: The trackers in the latest iPhone and iPod Touch are actually some of the best ones out there. Compass, gyros, accelerometers, the works. When implemented as a tracker in Unity, I do not notice any drift whatsoever, even when I try to fake it out by jumping and spinning in crazy circles.

Some newer Android phones have decent trackers, but the iPod Touch ones are fantastic, and you can get used ones for $100-150.

Post by **brantlew** » Sat Mar 17, 2012 10:37 pm

@Palmer: Whoa! I haven't heard that about the iPhone, but it's really interesting. And with the integrated GPS you might be able to do some really great things with free ranging VR. I typically avoid Apple products, but my wife has an iPhone 4, so I'm going to have to check that out.

Post by **pierreye** » Sun Mar 18, 2012 3:15 am

@Palmer - Thanks for the info on iPhone 4 as head tracker. I found out the software is called Control for IOS and can be used to script the format in OSC sending the info through WIFI so it's possible to read in GlovePIE. Could use the iPhone 4 as counterweight too for HMZ-T1.

http://charlie-roberts.com/Control/?page_id=19

Post by **CyberVillain** » Sun Mar 18, 2012 7:16 am

cybereality wrote:Hmm, the jitter in that video looks horrible. Nowhere near acceptable quality. However the end part looked OK, so maybe its just a bug with mouse emulation.

Thats raw output directly to freetrack (Arma supports freetrack and trackir)

Post by **CyberVillain** » Sun Mar 18, 2012 7:19 am

brantlew wrote:I think that was the raw signal without any low-pass filtering, so I'm not surprised by that type of base noise level. The bigger problem was how it seemed to swing to a completely new heading every 2 or 3 seconds. The other videos I have seen of this device haven't looked like that at all, so you gotta assume it's either a bum sensor - or else some kind of software glitch.

The big movements are me moving it (each 2-3 seconds), but observe, after each major move (by me) it continue to drift for a while until it stops.. And yes, its unfiltered

edit: also observe, it always drift in the opposite direction to the move

Post by **brantlew** » Sun Mar 18, 2012 8:24 am

There's also this app which can just output UDP over WiFi ! For a developer that's like music to my ears

http://itunes.apple.com/us/app/sensor-d ... 19802?mt=8

Post by **PalmerTech** » Sun Mar 18, 2012 11:28 am

Here is an app that was developed by some people I know, it showcases the tracker very well:
http://itunes.apple.com/us/app/fov2go-t ... 46367?mt=8

Post by **FingerFlinger** » Sun Mar 18, 2012 12:28 pm

Wow! I just tried this using a 3D mouse app, and it was almost like a plug and play headtracker.

@brantlew

Was just about to mention that app. I haven't tried it yet, but this is very cool!

Post by **brantlew** » Sun Mar 18, 2012 1:06 pm

Yeah I haven't gotten around to trying it yet, but it's something of a revelation. A high quality / packaged / near plug-and-play head tracking solution. And one that probably half the people already own! Shame on you Palmer for keeping this secret from us.

There's probably even a lot of easy mounting solutions with all the car cradle contraptions around - or maybe one of those belt clip thingies.

Post by **nrp** » Sun Mar 18, 2012 1:14 pm

[youtube-hd]http://www.youtube.com/watch?v=_A9kxrSPOSU[/youtube-hd]

I'm working on a tracker as well using open source hardware, firmware, and interface libraries. The design has a gyro, accelerometer, and magnetometer, a microcontroller for sensor fusion, PWM controlled RGB and IR LEDs, and a lithium ion chager, and it interfaces over USB, Bluetooth, and UART. On Bluetooth, the goal is to reach 10 hours of battery life with a 400mAh lithium polymer cell. The prototype in the video is 2"x2", but the next revision will be roughly half the size.

Post by **PalmerTech** » Sun Mar 18, 2012 1:43 pm

brantlew wrote:Shame on you Palmer for keeping this secret from us.

Hah, sorry! I really should have mentioned it, but it never occurred to me that you could pass the tracking data on to a PC. I have only used it as a counterweight on my PR4 before, driving the whole headset from a Unity environment rendered on the iPhone, and more recently as part of this project: http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=14658

NRP, that looks really cool. Any idea when you might have units we can try out?

Post by **nrp** » Sun Mar 18, 2012 1:57 pm

PalmerTech wrote:NRP, that looks really cool. Any idea when you might have units we can try out?

Right now my friend and I only have a few of them built, for the purpose of evaluating the sensors and sensor fusion algorithms. We'll probably be putting together around 10 of the second revision, and we'll be getting a few hundred of a third revision assembled towards the end of the summer.

Once the second revision is done, we'll probably distribute a few to people who can help develop the firmware and desktop software, or just work on interesting uses for it.

Post by **cybereality** » Sun Mar 18, 2012 1:58 pm

@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?

Post by **brantlew** » Sun Mar 18, 2012 2:07 pm

@nrp: Nice work!

Post by **nrp** » Sun Mar 18, 2012 2:11 pm

cybereality wrote:@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?

Sure. We're using ST's LSM303* 3 axis gyros and L3G* magnetometer/accelerometer ICs interfaced to an Atmega32U4 which is doing sensor fusion currently using Madgwick's AHRS algorithm and interfacing to USB using LUFA.

The main difficulty is that none of these parts are designed to be hand solderable, so it is difficult to prototype boards. I built a solder reflow oven for these. The software has been pretty easy due to the availability of good open source libraries that do much of the hard stuff.

You can follow our progress on the software and hardware at GitHub, and we'll probably be putting up a blog/website as things progess.

Post by **brantlew** » Mon Mar 19, 2012 1:18 am

Controlling SkyRim with my wife's iPhone.

A bit glitchy, so I'll have to get a video up tomorrow. Sleepy time...

Post by **CyberVillain** » Mon Mar 19, 2012 1:53 am

nrp wrote:
cybereality wrote:@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?
Sure. We're using ST's LSM303* 3 axis gyros and L3G* magnetometer/accelerometer ICs interfaced to an Atmega32U4 which is doing sensor fusion currently using Madgwick's AHRS algorithm and interfacing to USB using LUFA.

The main difficulty is that none of these parts are designed to be hand solderable, so it is difficult to prototype boards. I built a solder reflow oven for these. The software has been pretty easy due to the availability of good open source libraries that do much of the hard stuff.

You can follow our progress on the software and hardware at GitHub, and we'll probably be putting up a blog/website as things progess.

If its AHRS compatible it will also work with FreePIE software, or are you planning on having custom USB drivers on the PC side?

Post by **FingerFlinger** » Mon Mar 19, 2012 7:16 pm

@ brantlew

Did you already roll your own code using that SensorData app, or did you find something that works in the app store? I think that the Motion3DMouse app could work, but it crashes in 64-bit, so I can't really check it out.

Post by **brantlew** » Mon Mar 19, 2012 8:03 pm

I wrote my own interface to Sensor App via FreePie. The tracking is fine. A little bit of drift when I shake the phone around but still good. My only problem is that the refresh rate looks crappy - like 10Hz or less. I'm hoping just a software issue on my part. I'll post a video and code once I get it working.

Post by **FingerFlinger** » Mon Mar 19, 2012 9:45 pm

I thought that the app advertised 100Hz resolution? Is that not true for streaming? Either way, good job! It would probably have taken me a week or two to even get that far. (I'm not a great coder...)

Post by **brantlew** » Tue Mar 20, 2012 12:00 am

ok, here's the iPhone as a head tracker. You can see the sporadic tracking lag in the video. My current working theory is that it's something that the phone is doing. The sensor data is transmitted over WiFi so there are a myriad of things that I can imagine, but I think the phone is most likely. I turned off my firewall, VPN, and network apps to try and elliminate my computer as the culprit. This is my wife's phone so god knows what she's done to it, but she doesn't install apps so it's pretty much empty except the stuff that came pre-installed Anybody got ideas of what it might be?

Anyway, I think this is a good enough proof-of-concept to add the iPhone to our list of head-tracking devices. I'll release the FreePIE code sometime this week.

[youtube]http://www.youtube.com/watch?v=gQS2Tdo9ZVM[/youtube]

Post by **nrp** » Tue Mar 20, 2012 1:02 am

CyberVillain wrote: If its AHRS compatible it will also work with FreePIE software, or are you planning on having custom USB drivers on the PC side?

The interface is all serial and virtual (CDC) serial, so no custom driver will be necessary. We're writing C, Python, and potentially other libraries to make it even easier to access the data. Currently, we're just outputting a quaternion, since that is the smallest representation without singularities. The interface libraries will have functions/methods to access the data as Euler angles and rotation matrices as well.