TrackIR with Oculus

Post Reply
garrun
One Eyed Hopeful
Posts: 10
Joined: Tue Mar 19, 2013 2:12 pm

TrackIR with Oculus

Post by garrun »

Has anyone considered combining the head tracking of the Track IR with the viewing of the Oculus?

It seems to me like that might be a useful stop gap solution to working with 6DoF head tracking and lateral head movement tracking combined with Oculus optics. Perhaps the driver approach could work with TrackIR native games (Arma 2,various racing and flying games) to get a more immersive experience? Sounds fun to me...
User avatar
Evenios
Binocular Vision CONFIRMED!
Posts: 315
Joined: Sat Mar 16, 2013 11:02 pm

Re: TrackIR with Oculus

Post by Evenios »

i already suggested it. might be possible but very few TrackIR games support foward/backwards tracking so.. would be nice to try it in Flight Sim X as the vorpx drivers should work for that ;-) will have to see!
User avatar
TheLostBrain
Cross Eyed!
Posts: 100
Joined: Wed Feb 06, 2008 9:10 pm
Contact:

Re: TrackIR with Oculus

Post by TheLostBrain »

EDIT: Seeing your original goal was true 6DOF - hence the positional is important in this case. Apologies. :)
----------

Sure, could work pretty well in a 'limited movement' capacity.

Something that might offer a bit more flexibility could be to implement the Oculus tracker in FreePie:
http://www.mtbs3d.com/phpBB/viewforum.php?f=139

You could then setup a mapping between the Oculus tracker and the TrackIR and use the Rift semi-natively (minus positional) in TrackIR compatible games.
My Current VR Setup
- N-Vision Datavisor 80 HMD (1280x1024, 80 FOV at 100% Overlap)
- Ascension Technology Flock of Birds 6DOF Magnetic Tracking + Extended Range Transmitter
- Prototype HMD (~100 FOV) - Specs and design to be shared after patent issued.
- IZ3D for non stereo-ready apps
- GlovePie for TrackIR emulation for apps without native Ascension Tech FOB Support
http://www.thelostbrain.com/?tag=/head+mounted+display" onclick="window.open(this.href);return false;
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

The issue with TrackIR & Freetrack (FacetrackNOIR is out for obvious reasons) is the working volume and working angle limitations. When you're looking at a stationary monitor, you naturally keep your head facing the monitor and in a small area in front of the monitor. With the Rift, you are able and likely to turn your head a lot further, and move around a lot more. The tracking volume of the TrackIR camera is fairly small (~ 70° FoV IIRC), and if you turn your head more than about 45° away from the camera it loses track of the markers.
CyberVillain
Petrif-Eyed
Posts: 2166
Joined: Mon Jun 22, 2009 8:36 am
Location: Stockholm, Sweden

Re: TrackIR with Oculus

Post by CyberVillain »

EdZ wrote:The issue with TrackIR & Freetrack (FacetrackNOIR is out for obvious reasons) is the working volume and working angle limitations. When you're looking at a stationary monitor, you naturally keep your head facing the monitor and in a small area in front of the monitor. With the Rift, you are able and likely to turn your head a lot further, and move around a lot more. The tracking volume of the TrackIR camera is fairly small (~ 70° FoV IIRC), and if you turn your head more than about 45° away from the camera it loses track of the markers.
You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.

The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
daveo2013
One Eyed Hopeful
Posts: 27
Joined: Fri Mar 29, 2013 1:16 pm

Re: TrackIR with Oculus

Post by daveo2013 »

I'll definitely be trying mixing in TrackIR input when my rift arrives. The 6 DOF cockpits in FSX are really amazing with the TrackIR and I think tracking those extra DOFs might help with sickness as well as being spooky brilliant fun. The TrackIR does have the limitations mentioned above though, and probably heaps more latency than the rift's internal sensors. It offers the opportunity to prototype stuff immediately though. I would hope that a 6 DOF sensor is still high on Oculus's agenda. I'm probably more concerned about that than the display since better displays are certainly coming.
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.
The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!
The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
This is hopefully a given for all designed-for-VR games.
lmimmfn
Cross Eyed!
Posts: 138
Joined: Mon Apr 01, 2013 6:44 pm

Re: TrackIR with Oculus

Post by lmimmfn »

EdZ wrote:
CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.
The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!
The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
This is hopefully a given for all designed-for-VR games.
not really, because if im sitting in my nice leather chair that i normally use to game on my PC on, i know exactly where i am physically and i know my trackIR is right in front of me( but yes, i understand it wouldnt work if you were standing and rotating your body etc. )

Either way, the first thing im gonna do when i get my rift is play about with the SDK and TrackIR SDK and using Track IR when faced forward/slightly to the sides and use the rift for the rest, its a perfect combination. Lean forwards to look at something closer with TrackIR, look backwards with less accurate tracking whenever needed. TrackIR is a lot more accurate than the Rift, it has its limitations but why not use what it provides for cockpit games( removing the drift problem ) etc. and use the rift tracking for the rest.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: TrackIR with Oculus

Post by WiredEarp »

What we really need is a TrackIR -> Rift tracker fusion driver.
Something that would take both, and fall back/recalibrate the Rift tracker dynamically based on TrackIR's data.

Naturalpoint should really look at doing this. I think if they released a new version of their TrackIR software, that allowed this fusion, and also allowed use of the Rift in TrackIR games, they'd sell a lot of TrackIR's. They could still require TrackIR to be connected to use the software, even when using the Rift by itself, so it would basically become a hardware dongle. A few mods to the setup interface and we could be using NP software to tune our Rift setups easily.

If they don't look into something like this, or release a new version of TrackIR soon with multiple tracking cameras, integrated gyro, or similar, they will find their market drying up I think. Once games like flight sims start supporting the Rift, it will be a short step to users simply using Rift trackers to control their flight sims - which will eat their market otherwise.
CyberVillain
Petrif-Eyed
Posts: 2166
Joined: Mon Jun 22, 2009 8:36 am
Location: Stockholm, Sweden

Re: TrackIR with Oculus

Post by CyberVillain »

EdZ wrote:
CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.
The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!
The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
This is hopefully a given for all designed-for-VR games.
Your body is the reference point, so if your body is facing 20 degrees and your head 45 degrees then you are looking to the right 25 degrees relative to your body. I tried this with my Sony HMZ and Arma, and the only problem i found was that there is a small drift difference between the body and head tracker which causes the head to sway a bit. This is caused because TrackIR is relative to the body (0 is facing straight in the direction of the body) and in a 360 setup 0 changes so you have to sync body tracker and head tracker so that the TrackIR simulation recevies zero when head is not moving. Since the two IMUs never will be in complete sync the head will move a bit to the sides becasue of drift
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

I'm not talking about a software issue, I'm talking about a basic hardware limitation.
You put your TrackIR camera on top of a convenient stand, facing the chair you're sitting in. You look up to ~45° left or right, and everything works fine, and you have good track. Then you look at something more than 45° to the left or right. You lose track, and your viewpoint is no longer facing where your head is facing.
When you are looking at a monitor, this limitation isn't a problem, because you never need to turn your head that far, or you'll no longer be able to see your monitor. But with a HMD, you naturally turn our head a lot more and a lot further than if you were looking at a stationary monitor.
Commercial systems (e.g. Vicon) use additional marker constellations and multiple cameras in order to get around this limitation.
sepian78
One Eyed Hopeful
Posts: 1
Joined: Thu Jan 17, 2013 11:27 am

Re: TrackIR with Oculus

Post by sepian78 »

You could mount the TrackIR sensor on the ceiling and have the leds pointing upwards. The only thing you would have to do is remap the axis.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: TrackIR with Oculus

Post by zalo »

There is a solution to this. So simple, it's probably not even patentable.

Take a TrackIR camera and mount it on the ceiling facing down at the player. Mount two IR LEDs above the top left and top right corners of the rift. The camera should see two blobs. If it averages the positions of these blobs, it should get the XZ axises. The distance between the two blobs is proportional to the distance from the camera.

HOWEVER: Wonky distance values will emerge when the User tilts his head (causing the LEDs to appear closer together, and thus farther away).
SOLUTION: Use the Rift's IMU orientation to compensate for tilt interference. As sideways tilt approaches 90°, linearly scale the proportionality between: distance from the camera and distance between the blobs.

This will robustly get you XYZ position of the rift in 360° of yaw, looking straight up and straight down (pitch), and almost full tilt in either direction (the LEDs eventually overlap). It's fine though, because one is rarely ever able to tilt their head a full 90° in normal game scenarios.

BONUS: The angle between the blobs (camera screen space) will give you absolute yaw, so you can discard that old unreliable magnetometer. Tilt/pitch are referenced against gravity (accelerometer), so those will never drift.
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

Two LEDs is insufficient to fully describe 6DoF motion. It can reverse/rotate in all three axes without a measurable difference (left-right, forward-backward and up-down). Even three points is insufficient, and still reversible in one axis and rotatable in one axis. This is not an issue when using the TrackIR in front of a monitor, when the user self-constrains (seated, facing one direction). Even with the IMU providing two (or even three, once the magnetometer is integrated) axes, the pattern is still reversible in one axis.

By the time you add a large enough marker constellation to fully constrain tracking and is robust enough to survive multi-marker self-occlusion, you have a setup that is so far from the original TrackIR that you may as well use a whole new system and not have to worry about expensive cameras and proprietary interfaces. The two most obvious developments being the use of coloured markers, multiple cameras, or both.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: TrackIR with Oculus

Post by zalo »

You're right. Two LEDs by themselves are completely insufficient to describe hardly anything.

But when you use IMU data, it's possible to get absolute 3 DoF (position) with no reversible axes.

XY tracking is trivial, and you just use the screen space distance between the two blobs to get depth, and compensate for roll distortion with the IMU. As long as the camera is pointing straight down, the only user constraint will be that they can't do somersaults or cartwheels (or pat their rifts while rubbing their bellies).

Positional tracking of this nature will be sufficient to completely get of motion based simulator sickness.
If I can ever get FreePIE working properly, I'll make a script using a Wiimote and the Rift tracker to output it.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: TrackIR with Oculus

Post by WiredEarp »

. The two most obvious developments being the use of coloured markers, multiple cameras, or both.
multiple trackIR cameras seems the most logical way to extend TrackIR to support a much wider angle. Even with basic code, you could gain a much larger useable area this way.

@ zalo: something that combines gyros with optical realignment will probably be the best solution. I think Naturalpoint will need to look at something like this sooner rather than later.
Currently, I find my TrackIR is only good for a fairly limited set of angles. Even playing on a monitor, it can get annoying when you try to 'turn' the view towards the far edges.
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

I (and many others) are currently fiddling with multiple PS3eye cameras for marker tracking. Adding an IR filter (and removing the internal IR-block) gives you a camera with superior specs to the TrackIR webcams at a significantly lower price. There is also some experimentation over in the R&D forum of using a single PS3eye and multiple coloured markers surrounding the Rift, and of using a single camera and PS3move controller to provide 3DoF translational tracking (with the third axis coarsely determined by blob size).
Lilwolf
Cross Eyed!
Posts: 141
Joined: Thu Sep 02, 2010 3:08 pm

Re: TrackIR with Oculus

Post by Lilwolf »

I love the idea of combining the drivers.

but would trackir be the best solution? Seems like you could use a kinect in a corner of the room to recalibrate the input from the rift ever so often.

Take the rift input, say, they have only moved X distance in the last 10 ms (or however long the refresh of the kinect is) and know that the input 10 ms ago should have been there. compensate, re-calibrate and continue.

It would also be great if the rift drivers could take a calibration as input to it's drivers... so once you got it working, it would work with all rift games. but that's a stretch.
AussieMike
Two Eyed Hopeful
Posts: 62
Joined: Sun Dec 02, 2012 4:50 pm

Re: TrackIR with Oculus

Post by AussieMike »

instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
User avatar
Aeroflux
Binocular Vision CONFIRMED!
Posts: 217
Joined: Mon Jul 09, 2007 9:00 pm

Re: TrackIR with Oculus

Post by Aeroflux »

AussieMike wrote:instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
This is pretty much what I've been saying in the mek-controller thread, but there is dead silence in return. You could even mount leds in specific patterns for image recognition. I'd be happy with 180 degrees.
CyberVillain
Petrif-Eyed
Posts: 2166
Joined: Mon Jun 22, 2009 8:36 am
Location: Stockholm, Sweden

Re: TrackIR with Oculus

Post by CyberVillain »

ah, i thought where talking about using trackir games with 360 Imus sorry
EdZ wrote:I'm not talking about a software issue, I'm talking about a basic hardware limitation.
You put your TrackIR camera on top of a convenient stand, facing the chair you're sitting in. You look up to ~45° left or right, and everything works fine, and you have good track. Then you look at something more than 45° to the left or right. You lose track, and your viewpoint is no longer facing where your head is facing.
When you are looking at a monitor, this limitation isn't a problem, because you never need to turn your head that far, or you'll no longer be able to see your monitor. But with a HMD, you naturally turn our head a lot more and a lot further than if you were looking at a stationary monitor.
Commercial systems (e.g. Vicon) use additional marker constellations and multiple cameras in order to get around this limitation.
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

AussieMike wrote:instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
This is known as 'outside in' tracking, and is commonly done with a regular visual light camera and distinct markers. This is because you need some way to uniquely identify each marker in order to initially position yourself and position yourself after any loss of track (e.g. occlusion of camera). The TrackIR camera is not really suitable for this. There are also markerless implementations using various SLAM algorithms (Simultaneous Location and Mapping), but these currently have too high a latency for head-tracking use, and still have some issues with drift and spurious motion.

The TrackIR camera is a basic webcam with an nIR-pass filter, tied to some proprietary software. If you are not using the proprietary software it is merely a very expensive webcam, and there are better, cheaper choices you can modify (e.g. the popular PS3eye).
urban
Two Eyed Hopeful
Posts: 50
Joined: Thu Aug 23, 2012 8:31 am

Re: TrackIR with Oculus

Post by urban »

I think sensor fusion with the trackir would of been a great benefit to the oculus rift, as a lot of games have support for the trackir already built into them

It would also encourage the sim community to embrace the rift aswell, although I think they will anyway

The guy from trackir actually came to these forums and offered to help and seemed to have some good ideas on how to get the rift and track ir working together, but I got the feeling the trackir guys where not to welcome here on these forums

Shame really :(
lmimmfn
Cross Eyed!
Posts: 138
Joined: Mon Apr 01, 2013 6:44 pm

Re: TrackIR with Oculus

Post by lmimmfn »

urban wrote:I think sensor fusion with the trackir would of been a great benefit to the oculus rift, as a lot of games have support for the trackir already built into them

It would also encourage the sim community to embrace the rift aswell, although I think they will anyway

The guy from trackir actually came to these forums and offered to help and seemed to have some good ideas on how to get the rift and track ir working together, but I got the feeling the trackir guys where not to welcome here on these forums

Shame really :(
have you got a link to any of his posts?
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: TrackIR with Oculus

Post by zalo »

lmimmfn wrote:have you got a link to any of his posts?
lol here you go:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=15339
MemeBox
Two Eyed Hopeful
Posts: 55
Joined: Sat Sep 01, 2012 9:23 am

Re: TrackIR with Oculus

Post by MemeBox »

I have a prototype ready to go for when my rift arrives:

[youtube]http://www.youtube.com/watch?v=sEx1nNkEW_o[/youtube]

It uses 4 web cameras placed in the four corners of a room.
Any number of cameras can be added to improve the tracking.
Calibration of the cameras takes a couple of minutes and just requires the light source to be moved around within the space.
It is currently using a torch, but an IR source will work just as well.
It gives millimeter accuracy over a reasonable range at 30fps.

You will see some glitchyness in the tracking, I have ironed this out (mistake in the math) + camera bloom around the light source.

Combine this with rotation readings from the rift and if the tracking proves good enough you have full immersion...
EdZ
Sharp Eyed Eagle!
Posts: 425
Joined: Sat Dec 22, 2007 3:38 am

Re: TrackIR with Oculus

Post by EdZ »

Memebox: have you tried using a PSmove controller in place of the torch? It's not a point source, so you don't have the flare/glare issues, but you can still discriminate the blob down to a point reasonably accurately.
Post Reply

Return to “Oculus VR”