TrackIR with Oculus
-
- One Eyed Hopeful
- Posts: 10
- Joined: Tue Mar 19, 2013 2:12 pm
TrackIR with Oculus
Has anyone considered combining the head tracking of the Track IR with the viewing of the Oculus?
It seems to me like that might be a useful stop gap solution to working with 6DoF head tracking and lateral head movement tracking combined with Oculus optics. Perhaps the driver approach could work with TrackIR native games (Arma 2,various racing and flying games) to get a more immersive experience? Sounds fun to me...
It seems to me like that might be a useful stop gap solution to working with 6DoF head tracking and lateral head movement tracking combined with Oculus optics. Perhaps the driver approach could work with TrackIR native games (Arma 2,various racing and flying games) to get a more immersive experience? Sounds fun to me...
- Evenios
- Binocular Vision CONFIRMED!
- Posts: 315
- Joined: Sat Mar 16, 2013 11:02 pm
Re: TrackIR with Oculus
i already suggested it. might be possible but very few TrackIR games support foward/backwards tracking so.. would be nice to try it in Flight Sim X as the vorpx drivers should work for that will have to see!
- TheLostBrain
- Cross Eyed!
- Posts: 100
- Joined: Wed Feb 06, 2008 9:10 pm
- Contact:
Re: TrackIR with Oculus
EDIT: Seeing your original goal was true 6DOF - hence the positional is important in this case. Apologies.
----------
Sure, could work pretty well in a 'limited movement' capacity.
Something that might offer a bit more flexibility could be to implement the Oculus tracker in FreePie:
http://www.mtbs3d.com/phpBB/viewforum.php?f=139
You could then setup a mapping between the Oculus tracker and the TrackIR and use the Rift semi-natively (minus positional) in TrackIR compatible games.
----------
Sure, could work pretty well in a 'limited movement' capacity.
Something that might offer a bit more flexibility could be to implement the Oculus tracker in FreePie:
http://www.mtbs3d.com/phpBB/viewforum.php?f=139
You could then setup a mapping between the Oculus tracker and the TrackIR and use the Rift semi-natively (minus positional) in TrackIR compatible games.
My Current VR Setup
- N-Vision Datavisor 80 HMD (1280x1024, 80 FOV at 100% Overlap)
- Ascension Technology Flock of Birds 6DOF Magnetic Tracking + Extended Range Transmitter
- Prototype HMD (~100 FOV) - Specs and design to be shared after patent issued.
- IZ3D for non stereo-ready apps
- GlovePie for TrackIR emulation for apps without native Ascension Tech FOB Support
http://www.thelostbrain.com/?tag=/head+mounted+display" onclick="window.open(this.href);return false;
- N-Vision Datavisor 80 HMD (1280x1024, 80 FOV at 100% Overlap)
- Ascension Technology Flock of Birds 6DOF Magnetic Tracking + Extended Range Transmitter
- Prototype HMD (~100 FOV) - Specs and design to be shared after patent issued.
- IZ3D for non stereo-ready apps
- GlovePie for TrackIR emulation for apps without native Ascension Tech FOB Support
http://www.thelostbrain.com/?tag=/head+mounted+display" onclick="window.open(this.href);return false;
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
The issue with TrackIR & Freetrack (FacetrackNOIR is out for obvious reasons) is the working volume and working angle limitations. When you're looking at a stationary monitor, you naturally keep your head facing the monitor and in a small area in front of the monitor. With the Rift, you are able and likely to turn your head a lot further, and move around a lot more. The tracking volume of the TrackIR camera is fairly small (~ 70° FoV IIRC), and if you turn your head more than about 45° away from the camera it loses track of the markers.
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: TrackIR with Oculus
You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.EdZ wrote:The issue with TrackIR & Freetrack (FacetrackNOIR is out for obvious reasons) is the working volume and working angle limitations. When you're looking at a stationary monitor, you naturally keep your head facing the monitor and in a small area in front of the monitor. With the Rift, you are able and likely to turn your head a lot further, and move around a lot more. The tracking volume of the TrackIR camera is fairly small (~ 70° FoV IIRC), and if you turn your head more than about 45° away from the camera it loses track of the markers.
The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
-
- One Eyed Hopeful
- Posts: 27
- Joined: Fri Mar 29, 2013 1:16 pm
Re: TrackIR with Oculus
I'll definitely be trying mixing in TrackIR input when my rift arrives. The 6 DOF cockpits in FSX are really amazing with the TrackIR and I think tracking those extra DOFs might help with sickness as well as being spooky brilliant fun. The TrackIR does have the limitations mentioned above though, and probably heaps more latency than the rift's internal sensors. It offers the opportunity to prototype stuff immediately though. I would hope that a 6 DOF sensor is still high on Oculus's agenda. I'm probably more concerned about that than the display since better displays are certainly coming.
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.
This is hopefully a given for all designed-for-VR games.The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
-
- Cross Eyed!
- Posts: 138
- Joined: Mon Apr 01, 2013 6:44 pm
Re: TrackIR with Oculus
not really, because if im sitting in my nice leather chair that i normally use to game on my PC on, i know exactly where i am physically and i know my trackIR is right in front of me( but yes, i understand it wouldnt work if you were standing and rotating your body etc. )EdZ wrote:The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.This is hopefully a given for all designed-for-VR games.The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
Either way, the first thing im gonna do when i get my rift is play about with the SDK and TrackIR SDK and using Track IR when faced forward/slightly to the sides and use the rift for the rest, its a perfect combination. Lean forwards to look at something closer with TrackIR, look backwards with less accurate tracking whenever needed. TrackIR is a lot more accurate than the Rift, it has its limitations but why not use what it provides for cockpit games( removing the drift problem ) etc. and use the rift tracking for the rest.
-
- Golden Eyed Wiseman! (or woman!)
- Posts: 1498
- Joined: Fri Jul 08, 2011 11:47 pm
Re: TrackIR with Oculus
What we really need is a TrackIR -> Rift tracker fusion driver.
Something that would take both, and fall back/recalibrate the Rift tracker dynamically based on TrackIR's data.
Naturalpoint should really look at doing this. I think if they released a new version of their TrackIR software, that allowed this fusion, and also allowed use of the Rift in TrackIR games, they'd sell a lot of TrackIR's. They could still require TrackIR to be connected to use the software, even when using the Rift by itself, so it would basically become a hardware dongle. A few mods to the setup interface and we could be using NP software to tune our Rift setups easily.
If they don't look into something like this, or release a new version of TrackIR soon with multiple tracking cameras, integrated gyro, or similar, they will find their market drying up I think. Once games like flight sims start supporting the Rift, it will be a short step to users simply using Rift trackers to control their flight sims - which will eat their market otherwise.
Something that would take both, and fall back/recalibrate the Rift tracker dynamically based on TrackIR's data.
Naturalpoint should really look at doing this. I think if they released a new version of their TrackIR software, that allowed this fusion, and also allowed use of the Rift in TrackIR games, they'd sell a lot of TrackIR's. They could still require TrackIR to be connected to use the software, even when using the Rift by itself, so it would basically become a hardware dongle. A few mods to the setup interface and we could be using NP software to tune our Rift setups easily.
If they don't look into something like this, or release a new version of TrackIR soon with multiple tracking cameras, integrated gyro, or similar, they will find their market drying up I think. Once games like flight sims start supporting the Rift, it will be a short step to users simply using Rift trackers to control their flight sims - which will eat their market otherwise.
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: TrackIR with Oculus
Your body is the reference point, so if your body is facing 20 degrees and your head 45 degrees then you are looking to the right 25 degrees relative to your body. I tried this with my Sony HMZ and Arma, and the only problem i found was that there is a small drift difference between the body and head tracker which causes the head to sway a bit. This is caused because TrackIR is relative to the body (0 is facing straight in the direction of the body) and in a 360 setup 0 changes so you have to sync body tracker and head tracker so that the TrackIR simulation recevies zero when head is not moving. Since the two IMUs never will be in complete sync the head will move a bit to the sides becasue of driftEdZ wrote:The issue is, with the Rift on you cannot see where 'front' is very well. It is very easy to turn too far and lose track, and have you viewpoint suddenly come to a stop (or worse, reset to forward!) while your head keeps turning. Nausea city!CyberVillain wrote:You can still use the Trackir Protocol (or Freetrack) in for example Arma 2 and soon arma 3 to get headtracking and then 360 rotation from a seperate control / gun.This is hopefully a given for all designed-for-VR games.The important thing that Bohimia is doing right in Arma is that they have detached gun and head control, all games that want to aim at VR should to this.
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
I'm not talking about a software issue, I'm talking about a basic hardware limitation.
You put your TrackIR camera on top of a convenient stand, facing the chair you're sitting in. You look up to ~45° left or right, and everything works fine, and you have good track. Then you look at something more than 45° to the left or right. You lose track, and your viewpoint is no longer facing where your head is facing.
When you are looking at a monitor, this limitation isn't a problem, because you never need to turn your head that far, or you'll no longer be able to see your monitor. But with a HMD, you naturally turn our head a lot more and a lot further than if you were looking at a stationary monitor.
Commercial systems (e.g. Vicon) use additional marker constellations and multiple cameras in order to get around this limitation.
You put your TrackIR camera on top of a convenient stand, facing the chair you're sitting in. You look up to ~45° left or right, and everything works fine, and you have good track. Then you look at something more than 45° to the left or right. You lose track, and your viewpoint is no longer facing where your head is facing.
When you are looking at a monitor, this limitation isn't a problem, because you never need to turn your head that far, or you'll no longer be able to see your monitor. But with a HMD, you naturally turn our head a lot more and a lot further than if you were looking at a stationary monitor.
Commercial systems (e.g. Vicon) use additional marker constellations and multiple cameras in order to get around this limitation.
-
- One Eyed Hopeful
- Posts: 1
- Joined: Thu Jan 17, 2013 11:27 am
Re: TrackIR with Oculus
You could mount the TrackIR sensor on the ceiling and have the leds pointing upwards. The only thing you would have to do is remap the axis.
-
- Certif-Eyed!
- Posts: 661
- Joined: Sun Mar 25, 2012 12:33 pm
Re: TrackIR with Oculus
There is a solution to this. So simple, it's probably not even patentable.
Take a TrackIR camera and mount it on the ceiling facing down at the player. Mount two IR LEDs above the top left and top right corners of the rift. The camera should see two blobs. If it averages the positions of these blobs, it should get the XZ axises. The distance between the two blobs is proportional to the distance from the camera.
HOWEVER: Wonky distance values will emerge when the User tilts his head (causing the LEDs to appear closer together, and thus farther away).
SOLUTION: Use the Rift's IMU orientation to compensate for tilt interference. As sideways tilt approaches 90°, linearly scale the proportionality between: distance from the camera and distance between the blobs.
This will robustly get you XYZ position of the rift in 360° of yaw, looking straight up and straight down (pitch), and almost full tilt in either direction (the LEDs eventually overlap). It's fine though, because one is rarely ever able to tilt their head a full 90° in normal game scenarios.
BONUS: The angle between the blobs (camera screen space) will give you absolute yaw, so you can discard that old unreliable magnetometer. Tilt/pitch are referenced against gravity (accelerometer), so those will never drift.
Take a TrackIR camera and mount it on the ceiling facing down at the player. Mount two IR LEDs above the top left and top right corners of the rift. The camera should see two blobs. If it averages the positions of these blobs, it should get the XZ axises. The distance between the two blobs is proportional to the distance from the camera.
HOWEVER: Wonky distance values will emerge when the User tilts his head (causing the LEDs to appear closer together, and thus farther away).
SOLUTION: Use the Rift's IMU orientation to compensate for tilt interference. As sideways tilt approaches 90°, linearly scale the proportionality between: distance from the camera and distance between the blobs.
This will robustly get you XYZ position of the rift in 360° of yaw, looking straight up and straight down (pitch), and almost full tilt in either direction (the LEDs eventually overlap). It's fine though, because one is rarely ever able to tilt their head a full 90° in normal game scenarios.
BONUS: The angle between the blobs (camera screen space) will give you absolute yaw, so you can discard that old unreliable magnetometer. Tilt/pitch are referenced against gravity (accelerometer), so those will never drift.
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
Two LEDs is insufficient to fully describe 6DoF motion. It can reverse/rotate in all three axes without a measurable difference (left-right, forward-backward and up-down). Even three points is insufficient, and still reversible in one axis and rotatable in one axis. This is not an issue when using the TrackIR in front of a monitor, when the user self-constrains (seated, facing one direction). Even with the IMU providing two (or even three, once the magnetometer is integrated) axes, the pattern is still reversible in one axis.
By the time you add a large enough marker constellation to fully constrain tracking and is robust enough to survive multi-marker self-occlusion, you have a setup that is so far from the original TrackIR that you may as well use a whole new system and not have to worry about expensive cameras and proprietary interfaces. The two most obvious developments being the use of coloured markers, multiple cameras, or both.
By the time you add a large enough marker constellation to fully constrain tracking and is robust enough to survive multi-marker self-occlusion, you have a setup that is so far from the original TrackIR that you may as well use a whole new system and not have to worry about expensive cameras and proprietary interfaces. The two most obvious developments being the use of coloured markers, multiple cameras, or both.
-
- Certif-Eyed!
- Posts: 661
- Joined: Sun Mar 25, 2012 12:33 pm
Re: TrackIR with Oculus
You're right. Two LEDs by themselves are completely insufficient to describe hardly anything.
But when you use IMU data, it's possible to get absolute 3 DoF (position) with no reversible axes.
XY tracking is trivial, and you just use the screen space distance between the two blobs to get depth, and compensate for roll distortion with the IMU. As long as the camera is pointing straight down, the only user constraint will be that they can't do somersaults or cartwheels (or pat their rifts while rubbing their bellies).
Positional tracking of this nature will be sufficient to completely get of motion based simulator sickness.
If I can ever get FreePIE working properly, I'll make a script using a Wiimote and the Rift tracker to output it.
But when you use IMU data, it's possible to get absolute 3 DoF (position) with no reversible axes.
XY tracking is trivial, and you just use the screen space distance between the two blobs to get depth, and compensate for roll distortion with the IMU. As long as the camera is pointing straight down, the only user constraint will be that they can't do somersaults or cartwheels (or pat their rifts while rubbing their bellies).
Positional tracking of this nature will be sufficient to completely get of motion based simulator sickness.
If I can ever get FreePIE working properly, I'll make a script using a Wiimote and the Rift tracker to output it.
-
- Golden Eyed Wiseman! (or woman!)
- Posts: 1498
- Joined: Fri Jul 08, 2011 11:47 pm
Re: TrackIR with Oculus
multiple trackIR cameras seems the most logical way to extend TrackIR to support a much wider angle. Even with basic code, you could gain a much larger useable area this way.. The two most obvious developments being the use of coloured markers, multiple cameras, or both.
@ zalo: something that combines gyros with optical realignment will probably be the best solution. I think Naturalpoint will need to look at something like this sooner rather than later.
Currently, I find my TrackIR is only good for a fairly limited set of angles. Even playing on a monitor, it can get annoying when you try to 'turn' the view towards the far edges.
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
I (and many others) are currently fiddling with multiple PS3eye cameras for marker tracking. Adding an IR filter (and removing the internal IR-block) gives you a camera with superior specs to the TrackIR webcams at a significantly lower price. There is also some experimentation over in the R&D forum of using a single PS3eye and multiple coloured markers surrounding the Rift, and of using a single camera and PS3move controller to provide 3DoF translational tracking (with the third axis coarsely determined by blob size).
-
- Cross Eyed!
- Posts: 141
- Joined: Thu Sep 02, 2010 3:08 pm
Re: TrackIR with Oculus
I love the idea of combining the drivers.
but would trackir be the best solution? Seems like you could use a kinect in a corner of the room to recalibrate the input from the rift ever so often.
Take the rift input, say, they have only moved X distance in the last 10 ms (or however long the refresh of the kinect is) and know that the input 10 ms ago should have been there. compensate, re-calibrate and continue.
It would also be great if the rift drivers could take a calibration as input to it's drivers... so once you got it working, it would work with all rift games. but that's a stretch.
but would trackir be the best solution? Seems like you could use a kinect in a corner of the room to recalibrate the input from the rift ever so often.
Take the rift input, say, they have only moved X distance in the last 10 ms (or however long the refresh of the kinect is) and know that the input 10 ms ago should have been there. compensate, re-calibrate and continue.
It would also be great if the rift drivers could take a calibration as input to it's drivers... so once you got it working, it would work with all rift games. but that's a stretch.
-
- Two Eyed Hopeful
- Posts: 62
- Joined: Sun Dec 02, 2012 4:50 pm
Re: TrackIR with Oculus
instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
- Aeroflux
- Binocular Vision CONFIRMED!
- Posts: 217
- Joined: Mon Jul 09, 2007 9:00 pm
Re: TrackIR with Oculus
This is pretty much what I've been saying in the mek-controller thread, but there is dead silence in return. You could even mount leds in specific patterns for image recognition. I'd be happy with 180 degrees.AussieMike wrote:instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: TrackIR with Oculus
ah, i thought where talking about using trackir games with 360 Imus sorry
EdZ wrote:I'm not talking about a software issue, I'm talking about a basic hardware limitation.
You put your TrackIR camera on top of a convenient stand, facing the chair you're sitting in. You look up to ~45° left or right, and everything works fine, and you have good track. Then you look at something more than 45° to the left or right. You lose track, and your viewpoint is no longer facing where your head is facing.
When you are looking at a monitor, this limitation isn't a problem, because you never need to turn your head that far, or you'll no longer be able to see your monitor. But with a HMD, you naturally turn our head a lot more and a lot further than if you were looking at a stationary monitor.
Commercial systems (e.g. Vicon) use additional marker constellations and multiple cameras in order to get around this limitation.
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
This is known as 'outside in' tracking, and is commonly done with a regular visual light camera and distinct markers. This is because you need some way to uniquely identify each marker in order to initially position yourself and position yourself after any loss of track (e.g. occlusion of camera). The TrackIR camera is not really suitable for this. There are also markerless implementations using various SLAM algorithms (Simultaneous Location and Mapping), but these currently have too high a latency for head-tracking use, and still have some issues with drift and spurious motion.AussieMike wrote:instead of multiple trackir cameras (expensive) how about mounting a single trackir camera on your head and having multiple reflectors around you, mounted on poles
The TrackIR camera is a basic webcam with an nIR-pass filter, tied to some proprietary software. If you are not using the proprietary software it is merely a very expensive webcam, and there are better, cheaper choices you can modify (e.g. the popular PS3eye).
-
- Two Eyed Hopeful
- Posts: 50
- Joined: Thu Aug 23, 2012 8:31 am
Re: TrackIR with Oculus
I think sensor fusion with the trackir would of been a great benefit to the oculus rift, as a lot of games have support for the trackir already built into them
It would also encourage the sim community to embrace the rift aswell, although I think they will anyway
The guy from trackir actually came to these forums and offered to help and seemed to have some good ideas on how to get the rift and track ir working together, but I got the feeling the trackir guys where not to welcome here on these forums
Shame really
It would also encourage the sim community to embrace the rift aswell, although I think they will anyway
The guy from trackir actually came to these forums and offered to help and seemed to have some good ideas on how to get the rift and track ir working together, but I got the feeling the trackir guys where not to welcome here on these forums
Shame really
-
- Cross Eyed!
- Posts: 138
- Joined: Mon Apr 01, 2013 6:44 pm
Re: TrackIR with Oculus
have you got a link to any of his posts?urban wrote:I think sensor fusion with the trackir would of been a great benefit to the oculus rift, as a lot of games have support for the trackir already built into them
It would also encourage the sim community to embrace the rift aswell, although I think they will anyway
The guy from trackir actually came to these forums and offered to help and seemed to have some good ideas on how to get the rift and track ir working together, but I got the feeling the trackir guys where not to welcome here on these forums
Shame really
-
- Certif-Eyed!
- Posts: 661
- Joined: Sun Mar 25, 2012 12:33 pm
Re: TrackIR with Oculus
lol here you go:lmimmfn wrote:have you got a link to any of his posts?
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=15339
-
- Two Eyed Hopeful
- Posts: 55
- Joined: Sat Sep 01, 2012 9:23 am
Re: TrackIR with Oculus
I have a prototype ready to go for when my rift arrives:
[youtube]http://www.youtube.com/watch?v=sEx1nNkEW_o[/youtube]
It uses 4 web cameras placed in the four corners of a room.
Any number of cameras can be added to improve the tracking.
Calibration of the cameras takes a couple of minutes and just requires the light source to be moved around within the space.
It is currently using a torch, but an IR source will work just as well.
It gives millimeter accuracy over a reasonable range at 30fps.
You will see some glitchyness in the tracking, I have ironed this out (mistake in the math) + camera bloom around the light source.
Combine this with rotation readings from the rift and if the tracking proves good enough you have full immersion...
[youtube]http://www.youtube.com/watch?v=sEx1nNkEW_o[/youtube]
It uses 4 web cameras placed in the four corners of a room.
Any number of cameras can be added to improve the tracking.
Calibration of the cameras takes a couple of minutes and just requires the light source to be moved around within the space.
It is currently using a torch, but an IR source will work just as well.
It gives millimeter accuracy over a reasonable range at 30fps.
You will see some glitchyness in the tracking, I have ironed this out (mistake in the math) + camera bloom around the light source.
Combine this with rotation readings from the rift and if the tracking proves good enough you have full immersion...
-
- Sharp Eyed Eagle!
- Posts: 425
- Joined: Sat Dec 22, 2007 3:38 am
Re: TrackIR with Oculus
Memebox: have you tried using a PSmove controller in place of the torch? It's not a point source, so you don't have the flare/glare issues, but you can still discriminate the blob down to a point reasonably accurately.