Head Tracker Roundup

[cybereality]

@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?

[brantlew]

@nrp: Nice work!

[nrp]

@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?

Sure. We're using ST's LSM303* 3 axis gyros and L3G* magnetometer/accelerometer ICs interfaced to an Atmega32U4 which is doing sensor fusion currently using Madgwick's AHRS algorithm and interfacing to USB using LUFA.

The main difficulty is that none of these parts are designed to be hand solderable, so it is difficult to prototype boards. I built a solder reflow oven for these. The software has been pretty easy due to the availability of good open source libraries that do much of the hard stuff.

You can follow our progress on the software and hardware at GitHub, and we'll probably be putting up a blog/website as things progess.

[brantlew]

Controlling SkyRim with my wife's iPhone. A bit glitchy, so I'll have to get a video up tomorrow. Sleepy time...

[CyberVillain]

@nrp: That looks awesome! Can you explain a little about the parts you used and how much work was involved?

Sure. We're using ST's LSM303* 3 axis gyros and L3G* magnetometer/accelerometer ICs interfaced to an Atmega32U4 which is doing sensor fusion currently using Madgwick's AHRS algorithm and interfacing to USB using LUFA.

The main difficulty is that none of these parts are designed to be hand solderable, so it is difficult to prototype boards. I built a solder reflow oven for these. The software has been pretty easy due to the availability of good open source libraries that do much of the hard stuff.

You can follow our progress on the software and hardware at GitHub, and we'll probably be putting up a blog/website as things progess.

If its AHRS compatible it will also work with FreePIE software, or are you planning on having custom USB drivers on the PC side?

[FingerFlinger]

@ brantlew

Did you already roll your own code using that SensorData app, or did you find something that works in the app store? I think that the Motion3DMouse app could work, but it crashes in 64-bit, so I can't really check it out.

[brantlew]

I wrote my own interface to Sensor App via FreePie. The tracking is fine. A little bit of drift when I shake the phone around but still good. My only problem is that the refresh rate looks crappy - like 10Hz or less. I'm hoping just a software issue on my part. I'll post a video and code once I get it working.

[FingerFlinger]

I thought that the app advertised 100Hz resolution? Is that not true for streaming? Either way, good job! It would probably have taken me a week or two to even get that far. (I'm not a great coder...)

[brantlew]

ok, here's the iPhone as a head tracker. You can see the sporadic tracking lag in the video. My current working theory is that it's something that the phone is doing. The sensor data is transmitted over WiFi so there are a myriad of things that I can imagine, but I think the phone is most likely. I turned off my firewall, VPN, and network apps to try and elliminate my computer as the culprit. This is my wife's phone so god knows what she's done to it, but she doesn't install apps so it's pretty much empty except the stuff that came pre-installed Anybody got ideas of what it might be?

Anyway, I think this is a good enough proof-of-concept to add the iPhone to our list of head-tracking devices. I'll release the FreePIE code sometime this week.

[youtube]http://www.youtube.com/watch?v=gQS2Tdo9ZVM[/youtube]

[nrp]

If its AHRS compatible it will also work with FreePIE software, or are you planning on having custom USB drivers on the PC side?

The interface is all serial and virtual (CDC) serial, so no custom driver will be necessary. We're writing C, Python, and potentially other libraries to make it even easier to access the data. Currently, we're just outputting a quaternion, since that is the smallest representation without singularities. The interface libraries will have functions/methods to access the data as Euler angles and rotation matrices as well.

[CyberVillain]

Anyway, I think this is a good enough proof-of-concept to add the iPhone to our list of head-tracking devices. I'll release the FreePIE code sometime this week.

Nice work! Im a Apple hater so I will never use that hardware How did you execute the FreePIE plugin, did you do a generic UDP plugin or a specific plugin for the device?

If its AHRS compatible it will also work with FreePIE software, or are you planning on having custom USB drivers on the PC side?

The interface is all serial and virtual (CDC) serial, so no custom driver will be necessary. We're writing C, Python, and potentially other libraries to make it even easier to access the data. Currently, we're just outputting a quaternion, since that is the smallest representation without singularities. The interface libraries will have functions/methods to access the data as Euler angles and rotation matrices as well.

Ok, if your firmware is AHRS combaible it should work with FreePIE, if you wanna test it you can download binaries or source from https://sourceforge.net/p/freepie

[brantlew]

@CyberVillain: It's a specific plugin so the user doesn't have to implement parsing code in the script.

[pierreye]

Here's the extracted motionplus pcb in small package. With this size, it's possible to mount it inside HMZ-T1 or just stick under the HMZ-T1.

[space123321]

Impressive Pierreye - and excited! Will you be posting directions on the process you took for this mod?

[PalmerTech]

Brantlew, try restarting the phone (So that no apps are running in the background) and turn of cellular data.

There should not be any drift on the iPhone side, since it always knows which way is which. Try that Minus Lab app I linked to earlier, you will see what I mean.

[pierreye]

Impressive Pierreye - and excited! Will you be posting directions on the process you took for this mod?

I didn't take any photo but it's quite simple. Just unscrew the motionplus and take out the pcb. Cut of the connector and solder to the extension cable. I use a SDCard holder as the casing and hot glue the cover.

[brantlew]

Great idea for a cover!

[pierreye]

Used whatever resources we have. This is garage VR. Tested out the headtracker mounted under the HMZ-T1 and it work. At least it look much nicer and less nerdy compare to mounting the motionplus on the head

[PalmerTech]

Once the second revision is done, we'll probably distribute a few to people who can help develop the firmware and desktop software, or just work on interesting uses for it.

I would be love to be one of those people, if at all possible. I promise it will get used for interesting stuff!

[brantlew]

I finished up the FreePIE code for the iPhone tracker. It's uploaded it to FreePIE so you can download it, but it's not part of the build yet so I have an "unofficial" console-only build and test script if anyone wants to try it out. You will have to purchase the $9 Sensor Data App for your iPhone.
http://itunes.apple.com/us/app/sensor-d ... 19802?mt=8

Then just unzip this package, open a command console, and run "FreePIE.exe iPhone_test_script.lua"

FreePIEConsole.zip

Then under the Sensor Data App, configure the Unicast address to your computer IP, and click "Start Stream".

I was unable to fix or understand the periodic sensor lag problem. Tried shutting down all apps and disabling cellular radio but still no luck. I am curious if other people experience this. (Please let me know) Also, there is a bit of drift in the heading. I think if I calibrated with the magnetic compass, then that would fix it but I am only using the gyro-inertial values right now.

Good luck.

Edit: You will need to have the .NET 4.0 runtime installed and also the Visual Studio 2010 runtime. You can get that runtime here http://www.microsoft.com/download/en/de ... px?id=5555

[CyberVillain]

Once the second revision is done, we'll probably distribute a few to people who can help develop the firmware and desktop software, or just work on interesting uses for it.

I would be love to be one of those people, if at all possible. I promise it will get used for interesting stuff!

Me too, i would love to fix support in FreePIE for it!

[FingerFlinger]

@brantlew

Can you characterize the drift? Is it over time, or due to violent motion? I ask because I have an iPod touch, so it only has gyro/acc, and I cannot get it to drift at all, even by violently shaking it for 30 seconds. I'm using the 'iDesktop VR' app to test this, maybe its sensor integration algorithms are superior to those SensorData uses.

Anyway, I'll check out your script tonight and report back.

[brantlew]

@FingerFlinger: I actually notice it the most when I slowly rotate the phone 360 or 720 degrees. It seems to accumulate an error - maybe 5-10 degrees after a couple of rotations. As a first approximation, I am only using 1 set of sensor values so there is a really strong possibility that I can correct this error with the integration of more sensor data.

PS. It's possible that its error in my code, but I don't think so. I was careful to avoid coding practices that would lead to floating point accumulation errors.

[cbwan]

Hi,

we have already started a Google spreadsheet to classify the cheap trackers out there :

http://www.vrgeeks.org/wiki/device-classification

You are free to modify it

Also :

Fixed Position / Limited Angle Trackers

4. Playstation Move + Eye - The PlayStation Move controller can be mounted to the head and used as a head tracker. The API requires that the PlayStation console must be used as an interface bridge.

Actually you have unlimited head rotation, since this is done by the inertial trackers and does not require the blob to be viewed by the camera.

Fixed Position / 360 Degree Trackers

Trackers that allow full rotation but require the user to stay within a very limited space.

1. Razer Hydra - The Hydra device emits a magnetic field that can be detected by two controllers. Very accurate 6D positioning. (Are there calibration drift issues?). The controllers can be mounted to the head to implement head tracking but the device has a short range and is wired so free movement is limited. (What is the interface method?)

This is definitely not "fixed position", this tracker is real 6DOF. Within a limited range for sure, but all trackers have a limit

Unrestricted Motion Trackers

Trackers that allow for 360 rotation and unlimited (or very wide) range of movement. IMU devices fall into this category using both inertial/gravitational/and sometimes magnetic sensors to detect orientation and motion in space. Unfortunately these devices suffer from calibration drift and may need to be recalibrated quite often. Devices that only make use of inertial sensors are more susceptible to drift. Adding magnetic sensors generally improves the stability of the device but they are susceptible to environmental electro-magnetic fluxuations and can also drift.

Most inertial sensors don't work for translations at all, even if they have accelerometers. So I would place most of them in the first category.
Especially all the so called 9DOF IMUs, which in the end can only give usable 3DOF data..

cheers

[brantlew]

@cbwan:

we have already started a Google spreadsheet to classify the cheap trackers out there

Excellent! I'll take a look at it. I haven't updated this list in a while so it will help me fill in some of the details.

This is definitely not "fixed position", this tracker is real 6DOF

For my purposes, I consider it fixed position if you have to stay within a 3 meter radius (room size). I can see how there is confusion over this classification though. Maybe I should separate into two categories:
fixed = <0.5 meter radius (TrackIR)
limited = <3 meter radius (Hydra, Kinect)

Most inertial sensors don't work for translations at all, even if they have accelerometers. So I would place most of them in the first category.
Especially all the so called 9DOF IMUs, which in the end can only give usable 3DOF data..

I was careful in my definition to state that they were only able to detect motion. True, you can't reliably double integrate the IMU's to gather positional information. But you can use raw accelerometer and the first integral (velocity) to detect motion. This is often all you need to know. It is sometimes irrelevant to know if you traveled exactly 10 or 20 meters, but it is relevant to know that you started moving at about 1 m/s and then stopped.

[cbwan]

This is definitely not "fixed position", this tracker is real 6DOF

For my purposes, I consider it fixed position if you have to stay within a 3 meter radius (room size). I can see how there is confusion over this classification though. Maybe I should separate into two categories:
fixed = <0.5 meter radius (TrackIR)
limited = <3 meter radius (Hydra, Kinect)

Ok

Most inertial sensors don't work for translations at all, even if they have accelerometers. So I would place most of them in the first category.
Especially all the so called 9DOF IMUs, which in the end can only give usable 3DOF data..

I was careful in my definition to state that they were only able to detect motion. True, you can't reliably double integrate the IMU's to gather positional information. But you can use raw accelerometer and the first integral (velocity) to detect motion. This is often all you need to know. It is sometimes irrelevant to know if you traveled exactly 10 or 20 meters, but it is relevant to know that you started moving at about 1 m/s and then stopped.

(you also have to be careful to compensate for the acceleration that you detect while rotating, but that's another topic)

I'm not following the forum enough to know what your purposes are,
but I would advise a correct, precise head position.

Small and correct movements of the head are critical for presence because that's how you perceive naturally.
And if you have that, you don't even need to have a stereoscopic display to correctly perceive depth (http://cb.nowan.net/blog/2010/04/28/doe ... 3d-matter/)

We recently tried that at Laval Virtual. Some of the best French VR experts were on our booth and did not realize that we had used the Sony HMZ-T1 in mono.
They felt the 3d was very smooth and not giving any headache.

cheers!

[brantlew]

And if you have that, you don't even need to have a stereoscopic display to correctly perceive depth

I agree. I almost never use stereoscopic view because it's such a pain to setup and adds little.

[cybereality]

And if you have that, you don't even need to have a stereoscopic display to correctly perceive depth (http://cb.nowan.net/blog/2010/04/28/doe ... 3d-matter/)

We recently tried that at Laval Virtual. Some of the best French VR experts were on our booth and did not realize that we had used the Sony HMZ-T1 in mono.
They felt the 3d was very smooth and not giving any headache.

LOL! Maybe this was the Synopter effect ( http://www.newscientist.com/article/dn1 ... aches.html )?

[cbwan]

No it's not the Synopter effect
There are a lot of cues to perceive depth, stereosopy is only one of them.

Motion parallax is another very important one, and we had a good head tracker so the user could freely translate his head.

http://cb.nowan.net/blog/2010/04/28/doe ... 3d-matter/

Parallax, the apparent displacement of objects when you move, is another very important cue. Objects close to your eye move faster than objects that are far. And according to Laure Leroy’s PhD thesis (...), motion parallax (with head tracking) is at least, if not more, important than stereoscopy !

Monoscopic cues (perspective, atmospheric artifacts (fog), relative sizes, occlusion, parallax, shadows), and stereoscopic cues (eye accomodation and convergence angle, disparity between left and right images) are all very important depth cues.

The first simulator I worked with had a big monoscopic screen with head tracking, and this worked quite well too.

If your application requires distance or size evaluation, stereoscopy is probably required. Also know that it’s very hard to evaluate an absolute depth, but stereoscopy is very useful to evaluate a very small difference of depth. When seeing an object from 1m, we can perceive a 1mm depth difference. At 10m, we can only perceive a 10cm difference !

[PalmerTech]

Yeah, there are a lot of factors other than stereo that help perceive depth. That is a big reason trackers that can provide more than rotational data are so cool.

For some types of simulation, though, there might not be much of a difference. Racing, for example, where your head is not going to be sliding side to side very much.

[CyberVillain]

It wouldnt be of much use in a HMD were the image follow the head, and movies would have to be real time rendered.

[WiredEarp]

For some types of simulation, though, there might not be much of a difference. Racing, for example, where your head is not going to be sliding side to side very much.

It still makes a big difference being able to move your head about, even in flight sims (and I suspect racing ones). When I use a flight sim with only 3DOF, it feels really restrictive and unnatural after using 6DOF sims. Even though I only usually move it a few inches (except when trying to look behind me) it adds a lot to the immersion. Not to mention it also allows you to 'zoom in' on your instrument panel.

[CyberVillain]

When I play Arma I actually disables the zoom feature, it's annoying as hell, and in Arma it does not feel natural, maybe its better in other games

[WiredEarp]

I don't mean 'zoom' as in using the zoom function of the game, I mean the ability to put your virtual face closer to the instruments to seem them more clearly.

[CyberVillain]

I don't mean 'zoom' as in using the zoom function of the game, I mean the ability to put your virtual face closer to the instruments to seem them more clearly.

Yeah I know, but in arma it feels like you are zooming when you do it

[brantlew]

Anybody know anything about these sensors?

http://invensense.com/mems/catalog.html

[cybereality]

Anybody know anything about these sensors?

http://invensense.com/mems/catalog.html

Nice find. Looks promising, the the prices seem OK.

[bobv5]

All SMD as far as I can tell. Not a show stopper, but a pain to work with.

EDIT- cyber, your new avatar is quite disturbing.

[FingerFlinger]

Yeah, these will be a real pain to work with if you don't have a hot-air pencil.

[bobv5]

The hot air rework stuff isn't horribly expensive, I think the one I got was about £60. Cheap Chinese gear, doesn't actually meet the saftey laws, have heard stories of some potentially deadly wiring errors, works very well, but be careful what you buy.