IR headtracking?

[Sabre2552]

While thinking about the capabilities of wiimote and IR lights for headtracking, I started to wonder whether this could be implemented onto the 3D displays of today for added realism. an IR sensor could be placed on the monitor, and using IR lights on the polarized glasses, the location of the viewer's head could be tracked. Although new drivers would have to be coded for it and it would need the support of developers to implement this feature into their software, I think this would be a relatively cheap implementation of headtracking, and could be a useful tool available for curious and interested developers.

[cybereality]

Yeah, totally. I've been thinking about that myself ever since I saw the famous Johny Lee video. I guess the hardware is all there and relatively cheap. The TrackIR has been out for a while and is a lot more advanced than you can hack together with a Wiimote. That would work for custom applications, not really for commercial games though. Are you suggesting a stereo3d driver with built in head-tracking? Because that would be something else. Its still probably too niche for mainstream games, but that could be cool for some indie demos or something.

[Sabre2552]

I was thinking more that the technology could be implemented into the 3D monitor and glasses themselves in case developers want to take advantage of it. Some people would be deterred from the headtracking if they had to wear another device as well as 3D glasses. I thought this would get rid of one of the variables that would disencourage consumers, and also give extra justification to the glasses to help satisfy the people who think they're ridiculous and unnecessary. It's probably too niche currently, true, but if it were supported widespread... I think it would be a moderately cheap addition to the monitors/glasses, but could help get headtracking out into the public's eye. The S3D drivers could be like "ooo, look, S3D, awesome... and for a tiny bit of extra work, ooo, headtracking!!!", subconsciously making the developers want to make the game as best they could, and thus weaseling headtracking into the mix too.

[cybereality]

For headtracking like that to go mainstream I don't think it could use any IR stickers you had to put on your forehead. It would have to be based on facial-recognition or other computer vision algorithms. Something that can be used with a web-cam accurately. They already have some stuff like this (Sony EyeToy, etc.) but for some reason no game companies want to touch it. This game for PS2, Anti-Grav, had very good use of a camera for controls. If you moved your head you would see it move on screen. Moving one arm would make the character move one arm. It worked really well, you don't need any helmets or attachments. There are just no mainstream games that use it so most people assume it doesn't work. It works fine.

Slightly Off-Topic: Since we are talking about headtracking I though this would be a good place to bring this up. One thing that really interests me is the way the public has this aversion to S-3D, yet in other cases they seem to be all about it. For example, after that Johnny Lee video broke it seemed like the whole internet was buzzing with all this headtracking talk. Pretty much every comment you saw was saying how he was a genius and this was the coolest invention ever. Well, headtracking is nothing new. The TrackIR has been out for like forever, yet none of those mainstream people probably ever heard of it. Had it been a video about the TrackIR, people would have complained that you needed to wear glasses or a hat and said it sucked. In fact, I remember reading comments just like that in the past. So what I am getting to is that somehow Johnny Lee made headtracking "cool" just by integrating a Wiimote and posting a video on YouTube. I even recall people asking "how do you get it to look 3d like that" referring to the effect of the targets protruding out the screen. It was the faux-S-3D that really interested people, yet they don't even realize you can do that already with regular games. Its just interesting to me how this mob-mentality works. Like if someone could produce a Johnny Lee caliber video for S-3D it seems like things could just blow up over night. The biggest hurtle is that most people don't even realize S-3D exists!

[Sabre2552]

That's why I said integrate the IR into the glasses themselves. Though a webcam would probably give the most accurate results, yes. I would think that would cost more than IR, and it would depend on lighting conditions and a steady background image, though, wouldn't it?

Yes, I completely agree with you. The public needs to be shown S3D and headtracking in a "cool" way in order for it to be interested. I'm not sure how that we could do that, but if one of us could make that big of a spark for S3D, that would be great.

[cadcoke4]

I've experimented a bit with head tracking, and am back on the subject again to try to get it working with the SolidWorks CAD program I use at work.

Yes, the Johnny Lee video is what got me back on the subject. The method has been around for a long time. It always seems like someone at a university will do a proof-of-concept and then nothing else happens. The hardware for doing all of this is quite low cost, but the software never gets developed to bring this down to something the average user can implement.

To add some more info to that already posted;

FreeTrack is a open source program that works comparibly to TrackIR, with you building the LED headset yourself.

Perhaps 4 5 years ago I tried out a program called "Cam to Pan" which allowed you to implement mouse movement by slightly turning you head. I sort of got this to work with Inventor (another CAD design program) by starting the Orbit command and holding the mouse button down. I had to reverse the X axis and do some other scaling in the program, and it worked. But, it wasn't that good to be worth the trouble.

Joe Dunfee

[phil]

I like your thinking, Sabre2552 Head-tracking and S-3D are beautiful together! When you combine 'window' style head-tracking (like that Wiimote video) with - even crumby anaglyph, it's amazing how much it reinforces the stereo illusion. It's because, when you're viewing a stereo pair on a fixed monitor, even the tiniest shift in head position produces faulty parallax that the brain is more than capable of recognising as a sign of trickery. Precise window-style headtracking rips away that layer of artefact and your brain just says "yup, that stuff is really there, it's solid". This is something that I want more people to experience. Plus, it's just cool being able to peek round the edges of the monitor

I love the idea of a 3D display that has built-in headtracking 8) Especially if the hardware would do all the work of interpreting the head transform for you, so the game can just read a stream of numbers from the USB to know exactly where the player is (and roughly where they're aiming their head!)

Of course, the screens like WOWvx, that send out more than two images of the scene, don't suffer the same problem from small sidewards head movements - the parallax is as correct as the stereoscopy (within a certain movement range at least). Those formats have a totally different set of hurdles I suppose...

I'm going to go and polish-up the TrackIR headtracking in my demo now (thread). I'll post any improvements on that thread. See ya later!

[LukePC1]

3D display with headtracking? Isn't the Z800 doing exactly that? Or does it have only 3DoF?

I just wanted to say, that basically any HMD would be the perfect canidate for that - or is that just a Illusion that HMDs have some advantage above normal screens?

[yuriythebest]

3D display with headtracking? Isn't the Z800 doing exactly that? Or does it have only 3DoF?

I just wanted to say, that basically any HMD would be the perfect canidate for that - or is that just a Illusion that HMDs have some advantage above normal screens?

yes I'm very interested in that since I've never used an hmd except once in an arcade with some game with very primitive graphics and no noticeable stereo3d. I'm interested to hear from people who have used both a good hmd and screen based solution on which they would prefer to use. Do hmd's offer another lever of game immersion except for the fact that everything around you is dark when you are wearing one?

[phil]

LukePC1 - The Z800 is stereoscopic (frame sequential) and has 3 DOF headtracking (reads angles on 3 axes but no positions).

The problem I mentioned with head movements (faulty parallax breaking the 3D illusion) is specific to fixed monitors rather than HMDs.

I've got a Z800 and I really like it - except for the fact that it uses frame-sequential stereo I think you can buy a hardware mod that changes it to take two monitor inputs instead, but it requires you to remove the headtracker. The headtracker is really nice (video); can be a bit prone to drifting though.

crim3 has used the Wiimote for headtracking, in such a way that it continually calibrates the Z800 headtracker to remove any traces of angular drift, at the same time as providing full positional tracking - clever stuff! (thread).

[RAGEdemon]

I wonder if a hard mount would be an option - at least for the hardcore.

Something not quite as tacky as what follows:

A small piece of something is attached to the user's head (via glasses, headband, cap, whatever) which is linked via a non-flexible but very easily extendable (lengthwise) link (think radio antenna) to another place which is stationery, could be a table, a wall, even the shoulders. The head mount and the permanent mount are connected via the "link" with something resembling ball-socket joints.

Since there is a hard link from the head to the permanent "place", we can use something like encoders (3 perhaps, one for each axis).

Granted that this looks tacky, the major advantage here is that the head movements can be measured extremely accurately: there is no drift, no lag, no error, no calibration, no fancy driver compensation, or really any real kind of fancy programming needed.

One could use simple mouse emulation using just 2 axis to translate the head movements into very precise mouse movements much like the z800 software for maximum compatibility.

I have tried the TrackerIR. I know many people swear by it, especially those who play flightsims but I was left unimpressed due to the above mentioned downfalls.

The best one that I have used is the z800 but as a user mentioned earlier, it suffers from drift. Not only that but there is some lag, and the movement is not precise - when you move your head to the left, it goes towards the top left or bottom left depending on how it is feeling and the alignment of the planets. Not very much. It was good. Very good. But unfortunately, not good enough.

I have also had something from VRealities which was meant to be pretty good for its time. It was shockingly bad.

A physical solution, although ugly, would be a fraction of the price and the beauty would be in the simplicity and preciseness.

I did use accelerometers back while i was in education to make a head tracker, the mouse emulation code is quite simple.

At the time, what I learned was that it was very difficult to get anything like the accuracy of a physical mount connected via a physical link.

Perhaps someone with a bit more time and knowledge about the subject than myself would have more input into this kind of thing.

@Yuriy:

Hello mate,

The problem with HMD's is firstly they have a small field of view (the z800 and i-Glasses which I have owned looked like 21" monitor about a half meter away). Second, the resolution is poor - even with antialiasing, everything seems blurry - the world loses its life as all the textures blur together as the resolution can't handle anything crisp.

The major advantage is that the convergence and separation can be turned up very high compared to any other solution. There is also no ghosting which is fantastic.

For FPS, the z800, even with its low rez and small FOV, is unbeatable when it comes to immersion. The closest is a ghostless projected system.

Stay away from anything that comes from i-o displays though.

The biggest problem with stereo3D in general is the driver. I haven't read about a mod that lets the z800 accept dual input (I'd love to buy one again just to play with the iZ3D driver). nVidia driver is now useless for me and i'm sure for a lot of others, unless you are running Vista and have an 8+ generation card and have a zalman or have a red/blue glasses fetish.

z800 unmodded is dead without the nVidia driver, again unless you like to play old games on an old graphics card without any effects.

Currently, in MY OPINION (and please feel free to disagree), the best solution in terms of quality of stereo3D and driver support and future support, the iZ3D is very difficult to beat.

If the driver was a non-issue, then it would have to be a ghostless, flickerless projected setup or an HMD.

-- Shahzad.

PS, would someone please point me to this z800 mod. Has anyone had success with getting it to work with the Stereo3D God which is the iZ3D driver?

[Likay]

Think you're right on spot Rage and it doesn't have to bee too difficult or expensive either.

Another explanation: Using radiowaves for measuring distance do require some precise electronics (speed of light and centimeters). I had this idea before but i would start with an ultrasonic device (just an ordinary piezoelectric tweeter) connected to your head. Three microphones (in a triangular shape with, say 0.5 meters in between) are connected to a receiver/transmitter-box which contains all the necessary electronics.
It simply works like this: The receiver/transmitter-box sends pulses (wired or by radiowaves) to the ultrasonic device. Depending on position it takes different amount of time before sound reaches the microphones. With the delaytimes given you can calculate the exact position of the head. Beware that the signal has to be at a higher frequency that dogs/cats or other pets can hear.
Biggest problem is that i don't know anything about newer programming languages (read quickbasic ) lack of time and it stops there.

cheers

[phil]

Hi,

RAGEdemon - Here's that Z800 mod I mentioned:
http://www.tekgear.ca/index.cfm?pageID= ... ewproducts

I'm surprised that you've seen lag problems with the Z800 headtracker. I find it totally, perfectly responsive ("it's like it's not there"). But my only experience of the headtracker is from supporting it natively in my programming demo, reading yaw/pitch/roll values directly. Were you doing the same, or using its mouse emulation?

I wasn't too impressed with TrackIR to begin with, but the more I've spent time improving my support for it, the more I've become fond of it. And I'm delighted with the way it now enhances my stereo modes - compensating for the position of each eye in relation to both the 3D scene and the fixed screen that it's projected onto (v21, coming soon </plug>).

When I was doing that new TrackIR stuff recently, I found that the transformations required for 'window'-style headtracking are the same ones required for projecting stereo onto a single screen in such a way that the illusion is geometrically correct (making it life-sized and weighty). In version 21 of the demo you can see it's a big improvement over the separation+convergence model I was using before.