That sounds fantastic!! I think 1mm precision would be sufficient for just about everything - head, body, and props. Well I could maybe see using a local gyroscopic head tracker just for the added precision but it wouldn't be strictly necessary. Sucks about the FCC limitation for commercial sales, but I guess a home-brew unit that ignored the FCC could go pretty far There is some cool stuff you can do with redirected walking once you get about 100 ft or so. Is this intended for commercialization or is this just a side project? I would be super interested in either.

PS. I've been blasting signals from my home distribution amp out my HDTV antenna for years now - so suck it FCC!

I'm working on this myself as a hobbyist. I hope to sell it, but the price will keep it out of range of your average consumer. Ultimately, I would like to have some sort of laser tag setup where normal people can pay by the hour for time in it.

Well it sounds really interesting. How long have you been lurking here? You should consider posting a project thread in the DIY section. I would love to keep tabs on this. There are at least four of us around here working on natural motion VR systems and some cross-pollination might be beneficial.

I've been lurking for 4 years or so, just came back recently with all of the articles about John Carmack and PalmerTech's Rift. I'll definitely make a thread once I have something decent to show which should be whenever I get all of the bugs worked out of my latest board revision.

Cool - looking forward to it. One last question. Assuming you could transmit with as much power as you wanted, what are the practical range limits of the technology. Could you detect motion < 1cm at 100m ? Is transmission at that distance practical and if not are the limits technological, cost, or just regulatory?

Here's a paper where NASA did tests for a long range UWB tracking system. It says it's FCC compliant. The increase in distance is due to them using horn antennas with a narrower/more focused transmission beamwidth. It doesn't look like their accuracy is that great.

You could increase the range by decreasing the FOV of your antennas like in that paper, but then you need more antennas to cover the same area. If you were to increase the power, then you increase the complexity of your transmitter as you have to split the signal into multiple copies, amplify each copy a little bit, and then combine them all back together to get a single signal that is more powerful than the original. That's not easy to do with UWB signals, and then you'll be over the FCC limit.

If I could increase the power of my system, then I don't think I would have any decrease in accuracy as distance increased, but that's just a guess.

Were you guys aware of this? Am I just late to the party with this info? I know it's just Source games but that's pretty cool having direct access to the camera via an SDK! I've been looking for an easy platform to test full tracking with for a while (beyond just rolling my own Unity or Ogre project) and this seems perfect. Kudos to Valve!

https://developer.valvesoftware.com/wiki/Head_Tracking

Hello there, I’ve been redirected here from the Oculus Rift thread by Brantlew (thanks). I've read all the posts on this one and couldn't find a clear answer so I'll shoot!

I think full body motion tracking/positioning is a really interesting subject but as we all know it can get really complex, and even if all the technical issues could be sorted out it would still not be very practical in a home environment..

I'll try to simplify a bit and limit the scope of the problem to playing a FPS game, sitting down on a fixed spot and using a gamepad to move the character's body, run, jump, shoot, etc (like in Carmack's E3 demos).

So it seems like even in this relatively simple configuration the issue is still being able to track the HMD/character's eyes position in relation to the center of the body, so we can do small strafe movements with the head/torso (to dodge fireballs for instance) or tilt slightly forward to look around corners, etc. Small details you may say, but something that in my opinion could make the whole VR immersion work (or fail).. All this because apparently gyroscopic sensors like the Hillcrest are no good to track total positioning.

In this context could we not simply attach a few stickers/leds on the Rift and use an optical device like Kinect to get a pretty accurate x,y,z eye position, while letting the Hillcrest tracker get the pitch-yaw-roll of the head? As I understand Kinect has a pretty low latency if you ignore the whole skeletal tracking (not necessary) and just get the depth map and the rgb image. You could tell the user to sit still before starting the game to get the initial position, and then just map the new relative positions with the game engine..

Could this not be a fairly simple solution where we would still get a much more immersive response than with a single gyroscopic sensor? Or am I missing something? Has anyone actually tried to integrate something similar in a working engine?

And even if it is not as simple as i make it sound.. should we not focus on solving this before anything else?!

In a nutshell - yes. Optical tracking is probably the most straight-forward solution to the 6DOF problem. Take your pick - Kinect, Move, webcam, or even Wiimote. Any of these could be retooled for fixed-position 360 degree tracking - assuming you are willing to setup and calibrate an area with multiple cameras or markers. The proof-of-concept is in cinema and game motion capture which all employ this technique. It's just that nobody has really successfully completed and packaged this solution yet for general consumption. Maybe because all the necessary and inexpensive equipment has only recent become available? The Holodeck project looks like a promising experiment into this and there are few small projects scattered within this forum and academia that try to address it. But nothing yet has really made it out of the "lab" into user's hands yet. But it feels like things are starting to accelerate in this direction.

That's what Carmack is talking about. We're at an inflection point and now with immersive HMD's finally within our grasp, people are going to start going wild with VR applications. And a few hobbyists can make a huge difference

Thanks again brantlew, I understand, but my point is: in the simple configuration i described before (which for me should be the starting point for VR games) where no 360 degree tracking is needed, should a single Kinect camera not be enough to determine the eye's relative 3D position to the initial point? Which in turn would simplify a lot the coding/integration?

There could be markers on the sides of the Rift as well, so position could still be determined when not facing forward..

Without the need for 360 tracking, there are already optical solutions like TrackIR and FreeTrack that can accomplish this. A few flight sims already support 6DOF cockpit views this way. But there are some hard limits on the yaw angle that it can track.

Placing markers around the head is an idea that has been bounced around a good bit, and I'm surprised nobody has implemented that yet. It does make the problem a bit more complex, but if you augmented the algorithm with a gyroscopic sensor then it seems like a very workable solution to me and would fix the yaw limit.

There is a small issue with the Z-axis on front facing cameras though. They have to use perspective transforms to calculate the Z coordinate which is less accurate than the X and Y axis which they can retrieve more-or-less directly. So your side-to-side head motion will be tracked more accurately when facing forward than when looking at say 60 degrees off center. I don't know how much of a problem in practice this would be but it's there in principle.

I think a 2-camera setup with a wide stereoscopic view of the player would be a good solution. Triangulation of the markers would provide far more accurate results in all dimensions. Combining that with a "ring" of markers on the head and you could build a pretty damn good tracker (says me).

And for completeness sake - you can also invert the solution. Mount either a mono or stereoscopic camera (Kinect) on your head and track the environment instead. There are advantages/disadvantages to either approach.

Cool sorry for my ignorance, I didn't know about TrackIR and FreeTrack.. I can see that the TrackIR 5 supports up to 120 FPS and it's "only" 150$, not bad.. and FreeTrack is an open-source Library for Windows that supports TrackIR, perfect match!

The question that comes to mind is: why are Carmack and Palmer playing around with gyroscope sensors like the Hillcrest then, instead of with something like the TrackIR 5? (or in combination with it). That would solve many of the problems that the prototype E3 demo had, as Carmack himself admitted in some of the interviews.

I can see why a solution similar to the TrackIR has not been used in FPS games until now, but I think that in a VR environment it makes a lot of sense. In fact I believe it could work really well, and is something that we could enjoy right now (without too much trouble it seems) and would set a standard for future FPS games VR integration.

Anyway I don't think that Carmack has the time to read this forums anymore.. I guess I could try Twitter. Or if you go to the Quakecon VR panel maybe you could ask him yourself Can we expect 6 DOF support via optical sensors for Doom 3 BFG / Doom 4!?

My guess is because it's just simpler to implement and integrate a gyroscopic tracker quickly for the demo. For 3DOF, gyroscopic trackers are really great solutions. They are accurate, responsive, flexible, and require practically no calibration. You just plug it in and go - perfect for a quick demo on the road.

Also from the get-go, the Rift was designed as a 360 experience. That's why he's shipping the kit with a wireless HDMI transmitter. So you can be unbound by cables and spin freely. With that in mind there is no optical technology that can support that right now. So it's simpler just to throw a tiny little gyro tracker in with the kit as a reference design than to try and whip up and test a multi-camera solution. It's up to us now to work that out. If by next year, someone hasn't implemented a multi-camera and/or multi-marker version of FreeTrack then I may just take a crack at it.

Thanks for the reply Brantlew, it makes sense.

Still I'm quite surprised that I seem to be the only one around here that thinks 6 DOF optical tracker support for Doom 3 BFG would be the way to go, even without the 360 degrees freedom (in a controller-in-hand, sitting-on-the-sofa setup). Basically the controls would be exactly the same as when you play the console version, with the only difference that you would BE THERE.

After that is really working then we could move on to the next step, like integrating the Leap or the Hydra to replace that ugly xbox controller

Well it would have been great if Carmack put "hooks" into the game so that we could plug into the camera and/or player model and control it via third party control devices (and he may still), but I can understand why he would not expose that type of functionality without a reference design to test with. So even if we do come up with good 6DOF trackers we have a serious lack of software support. There's no interface for general camera translation like there is for camera orientation (mouse). You can sort of "hack" it by tapping into the rendering pipeline and offsetting the camera (see cybereality's 3D "roll" driver) but it's not the best solution. So we're left with either custom and open source game engines or potentially Valve games (that's why I got so excited about 10 posts back when I found the Valve SDK!). Honestly that's probably why nobody has pursued this heavily. Even if you built a fantastic 6DOF tracker, what can you do with it? Research mostly. But for gaming you're stuck with flight and car sims.

That's precisely why i think it would be great if Carmack (or someone from ID) tweaked the engine to support this type of optical tracking and camera translation input (available right now with a commercial product like the TrackIR 5). Then at least we would have one great example of how 6 DOF can work with FPS games and a HMD. If this was a success like I think it would, then other future games and engines would support it as well.

It may be a bit too much to ask, but I think now is the right time to do it!

Well if the Valve SDK actually works the way they say it does, then we can implement perfect tracking examples in some pretty good games. Just need the Rift warping drivers...(hint hint Cyber or Emerson)

While I have generally thought camera tracking was the way to go myself, I know that one of the main problems with that technology is latency. You have to get the image from the camera over a USB port, and then process the image.

Probably the ideal version is an integrated camera and processor, like what the Wii remote is, but with greater resolution and field of vision. Then there would have to be some higher level connection between multiple cameras to integrate the results which could probably be done on the main computer.

Joe Dunfee

You can sort of get 6DOF with intercepting DirectX calls, but it comes with problems. Mainly I have not found a way to alter the view frustum yet, so objects and areas of the screen are clipped and/or culled prematurely. In theory you should be able to move the camera's position (which is what the stereo drivers are doing) but you can only move so much. For example, if you tried leaning around a corner, the game would probably think the area beyond the edge was not visible and thus would not be shown. But I am still investigating this scenario.

I think the problem is back face and occlusion culling, which is handled by the application, not the graphics engine.
Oh how I dream of a world where all games have support for this implemented!

Well I haven't been able to find out the exact latency of the TrackIR 5 after image processing but 120 Hz refresh frequency is promising. From all the reviews i have seen it seems to be very low.

The fact that none of the reviewers complained about latency is a good sign as well, especially if you take into account that people use it mainly for flight simulators and driving games.. a type of gamer who tends to be very picky with this sort of thing.

It would be interesting to know if it could get the Carmack low-latency seal of approval

Chriky has some practical experience with camera latency with his little project. He claims his image processing can mostly keep up with the raw Move camera feed of 187 FPS. I don't know what the initial latency is but I suspect it's pretty low. Maybe not Carmack-low but he's aiming at perfection. I think you can still have some pretty profound experiences running even twice as slow as the Carmack-rate.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15064

OK I've just sent a PM to Mr Carmack mentioning this thread and asking for 6 DOF tracking support in Doom 3 BFG / Doom 4..

I think it's a valid request if he really wants to push VR forward, so, you never know

Just been catching up with this thread, a lot of great ideas in here. Optical still seems like the way to go but RF sounds really interesting to, the more options we have the better really, which got me thinking about ultrasonic range sensors, has anyone done any work with that? You can get dirt cheap "parking sensors" that are supposed to have ~10m range with 1mm accuracy, so in your typical sized (square-ish) domestic room couldn't we use a couple of them at 90 degrees and calibrate from a known starting position? A good yaw reference still seems to be a major problem, or are there any optical / electronic "compass" sensor packs that can do a decent job? I'm still getting up to speed on all this, and really don't have the math abilities to work out the integration etc, but I just wanted to throw ultrasonic sensors into the mix as they might be a good fit for certain environments, any thoughts guys?

@Nick3DvB: I don't know anything about them, but I've always been curious about them myself. It seems like they would have a lot of the same good properties of RF, but without the government regulations. I'm not sure how much of an issue occlusion and multipath would be, but it seems it could be a fruitful area of research. If you find some good info, you should post it.

Will do, I'm looking at robotics sensors at the moment:

http://www.parallax.com/Store/Sensors/O ... fault.aspx

http://www.robot-electronics.co.uk/acat ... ngers.html

http://www.micro-examples.com/public/mi ... anger.html

Still trying to find some good real-world numbers on the accuracy / range, the low range ones really are dirt cheap!

http://www.ebay.co.uk/itm/HC-SR04-Ultra ... %26ps%3D54

Well the price is right. Would be interested to see what you could do with one of those.

Sonar is too slow. The width of the actual signal is very long. That means that reflections can easily interfere with the direct path signal. Accuracy also decreases as the rise-time of a signal decreases (less certainty in exactly when the signal crossed a certain threshold because it does so much more slowly). The update rate is also slow because you have to wait for this long signal and its reflections to completely clear the room before you send the next sonar chirp. You're also incurring latency because the speed of sound is ~1 ft/ms. For every foot of distance from the transmitter, you're incurring an extra millisecond of latency (head sends signal, head continues moving, receiver picks up signal, head has had milliseconds of movement since the signal was received). UWB RF is like sonar but a billion times faster (picoseconds vs milliseconds). You're better off sticking to optical tracking where you don't have to worry about a time of flight system.

Edit: Here's a company that sells systems that do ultrasonic positioning via nodes you place around a room and a wearable transmitter on the moving target: http://www.hexamite.com/

Thanks Krenzo. Makes sense that the speed of sound would be a huge obstacle.

The slow speed of sound is actually what makes it so popular with robotics and other low cost equipment since you don't need a fast CPU to get a good range measurement. A one microsecond (1 MHz) update rate can get you better than an inch of accuracy which is very good for navigating an unknown environment.

Thanks for the explanation, I thought there must be a reason nobody had done of this before, the laws of physics are a pain sometimes.

But maybe they could still be used in conjunction with accelerometers? (as a reference with a relatively low sampling rate)

RF clearly has a massive speed advantage, I remember playing with a Theremin at a museum once, I was amazed how responsive it was.

You can use anything in conjunction with accelerometers, but sonar doesn't have anything going for it over optical, magnetic, RF, or other positioning systems. The powerglove used ultrasonic positioning, and that didn't work out so well. The original/high end powerglove used magnetic positioning which worked well, but they went with ultrasonic positioning to reduce cost. It resulted in many problems (eg reflections from the wall behind your TV) that turned the powerglove into a poor product. If you want to experiment with ultrasonic positioning, buy a powerglove off of eBay and stick an accelerometer on it. I would instead suggest you experiment with magnetic or optical tracking.

Edit: Here's that ultrasonic positioning system I mentioned in my previous post: http://www.hexamite.com/

Thanks again, sounds like a basic sonar system is out but I'm not ready to give up on a sonic solution just yet, my other idea was to emit constant ultra-sonic or sub-sonic tones from your standard 7.1 surround system (or special high range speakers) and use several stereo / positional mics on the player to derive orientation (through a basic volume analysis or something more complex involving different frequency tones from each speaker). It may even help with the yaw problem, and no time of flight issues, but plenty of others I'm sure...

You might be able to do this with a single microphone as well, and make calculations based on the doppler effect.

Doppler effect is usually used for measuring relative speed (as far as I'm aware), so not sure that it makes sense for getting an absolute position. I think that multiple microphones (and/or sources) and measuring phase shift difference is the best way for an absolute position.

Yes, I was really thinking this would be a reference system in conjunction with accelerometers / gyros again, for the sub mm stuff.

I'm no acoustics expert so I'm sure there are all kinds of things you could do with doppler, phase, interference effects and other wave dynamics that I have no clue about.

I'm not sure here, but taking in account the doppler effect allows you to find the distance of the sound source (thats how the distance of stars is known). I just figured if one has three sound sources you could do triangulation with that. But I 'm no expert by far.

Doppler is great over astronomical distances but I'm not sure on this scale, given the sensitivity needed, but I'm no expert either.


EDIT @ Kajos:

Looks like you were onto something with the Doppler effect!

http://fr.twitter.com/ID_AA_Carmack/sta ... 4254764032

The measuring of the distance to stars is a little different since the relative motions are either consistent or calculable. They use the change in Doppler effect when the earth is moving at a known speed perpendicular to the light coming from the star. I don't think that it has much use where the position is as random as something controlled by the user, and will certainly not be of use when all parts of the system are stationary.
There is a good paper on this:
https://pumas.gsfc.nasa.gov/files/06_09_05_1.pdf

I was thinking a basic volume analysis on 4 speakers should give pretty good results, as a starting point anyway,

I'll have to start experimenting in the regular audible range and hope the frequency shift doesn't matter too much,

we'd want to pick an inaudible tone range that doesn't impact on the audible range sound quality too much either.