I've been thinking about this--I think my first attack on the problem will be to keep it simple: I'll try to render the initial buffer prior to warping at 1280x1600 and effectively try to do an old-school FSAA approach to smooth things out a bit. If that works, great, otherwise I'll have to start digging into the more complicated algos.

@Emerson:

Looks great, but we've to find a solution for the heavy aliasing.

spyro

I've been using a more hacky solution of rendering each eye out to a 1024x1024 texture, then using them to texture actual 3D distorted planes which are in front of the main camera.

So, two 'eye' cameras with 4:5 aspect ratio rendering to 1024x1024 textures, then one 'main' camera with 16:9 ratio just looking at some sphere segments textured with the 'eye' images.

@Emerson: Your solution is looking better and better. Thanks a bunch for posting the warping equation, this will help me a lot.

@brantlew: To be honest, I'm not sure the AA is needed. I know at least with the older prototype Palmer sent me, there was a heavy diffusion layer on it and the image had a soft look to it. If the final Rift is anything like this, then it will be a little blurry anyway, so I don't think the aliasing will be a problem. Though you can still notice pixels somewhat, so I guess an AA pass could help fake a higher resolution.

@cyber: Yeah just like all AA, it's not strictly necessary although on the Rift it's a little different because different parts of the screen will look more aliased than others. Of course the worst aliasing is towards the edge of your vision so less of a concern. If the only solution is to render at high resolution and down sample, I would rather not take the performance hit for it. But if there is a simple way that doesn't add too much overhead, it would be nice.

I think I remember reading Palmer ditched the diffuser because of the loss of contrast?

Good call IMHO, people can add one themself if they really want to give their Rift cataracts!

Maybe Nvidia's new FXAA modes could help us? Interesting bit on temporal AA here:

http://timothylottes.blogspot.co.uk/201 ... -info.html

One quick fix that I found--I was using point sampling on the intermediary texture, not lerping--it helps a bit:

That looks a lot better. Perfectly playable as far as I'm concerned.

Now I just need to grab a Hillcrest tracker, whip up the FreePIE integration code, and we're all set for 2 games once the Rift bundle comes out.

Hmm, too me it looks like only the "frame" thats warped?
Look at the ATM machine in that pic, it's lines are still straight?

Nah, it's just subtle (moreso because the area of highest distortion is inside the black-bar area in ME).

Same area, just with the camera pointed further down:

@Emerson: So are you done with this mod? (beyond testing and verification on an actual Rift) I don't know how much time you put in, but it seemed really fast. Amazing work! Do you have plans for conversion of any other titles?

That looks awesome Emerson! I just tried the cross-eye test, holding face about 1 foot away from the monitor and this is going to be unREAL! So pumped!

I think I'm gonna call this roughly feature complete for the first round--I'll probably try to spend the rest of the week cleaning up some of the more dodgy stuff I did in the code, and maybe throw it against some other games to see how much work it would be to support a few more. Hopefully I can get it out and let other people play with it this upcoming weekend.

Once that's out of the way I'll start poking around with additional functionality (maybe the screen-resolution-adaptor behavior mentioned earlier in the thread, or see if I can create a usable roll-axis).

Sounds great.

One problem I see with roll is that there is no native command interface for it. Yaw and Pitch can be mouse emulated, but if you provide roll you will also need to provide a method for controlling it. If you are just supporting the Rift kit then you could poll the Hillcrest tracker directly, but I would suggest providing a generic interface (socket/pipe/Window Messaging,etc..) so that other trackers could be attached to it. I plan on putting Hillcrest support into FreePIE in the next few weeks so in theory we could come up with a scheme to drive the "roll interface" via FreePIE and support a wide range of trackers including the Rift.

Yeah, clearly visible there, cool!

I've noticed so far that the images are rendered at traditional 640x800 resolution then distorted, with the widest point aligned to the narrowest part of the Rift FOV, leaving the large black borders.

Perhaps we should be rendering to a larger FOV at higher resolution and crop some image off the top and sides but fill in more of the black area? Given the whole point in the Rift is the large FOV I don't think those corner pixels should go to waste (although I understand some will be physically obscured to improve the immersion).

Know I've been kind of quiet lately, but I've still be hacking away at the driver. Now I have fixed the camera separation controls (previously it was skewing the view at a strange angle) and I have also got convergence working. Will probably add the side-by-side viewing over the weekend and finally get to try this out with my HMZ-T1. I'm excited.

Rockin! Almost there. We may have several Rift-ready titles by August.

Nice work Cyber.

Could the camera control code work with other Source games, HL2 etc?

Sounds great CyberReality! Exciting to think we will actually be able to use our Rifts for more than one game when we get them!

This is excellent Cyber! ( why do we have so many separate DIY forums: p)

I found an interesting hack that some one did. They modify Diablo 3's camera directly in memory to zoom and rotate. It's an interesting way to solve this since, besides not disrupting shaders, it can potentially take advantage of frustrum culling, allowing for head tracking. Of course it would be difficult to render twice.

http://www.youtube.com/watch?v=0J22sIG- ... ata_player

While I think the direct memory editing might end up being the best looking solution for rigging direct linkage of camera and a 6DOF tracker, due to the above mentioned culling issues, I also think it's likely to be the most fragile. The memory locations for this sort of thing are going to be game specific, and probably even different patch-to-patch on the same game.

I'm hoping my DirectX intercept solution will generalize to a wider variety of games with only minor tweaks necessary. I've been trying to get it to work with something other than Mirror's Edge, and there's certainly differences from engine to engine, but I'm making progress on a solution that should adapt well. (Even commercial offerings like IZ3D and DDD need game-specific profiles, though, and I'd expect the same here.)

Ideally, one could use both--the DirectX intercepts would render whatever camera views actually come down in 3D, and the direct memory editing would make sure that that camera is tracking the HMD view accurately.

Why is it so hard to do roll? Surely if you have access to the matrix you can do any transformation?

Roll is feasible because it mostly doesn't expose any new geometry that you weren't already seeing--just rotating the existing stuff (there's a little new geometry from circling the square, so to speak). That still leaves all the head-translations, too, though, which might run into culling problems (most games will dump a significant portion of their geometry, as much as they can, before even talking to DirectX at all).

From an optimal experience perspective, it'd be best to be able to manipulate the camera data structures directly, because you could skip the sensor->mouse emulation->direct input->game layers and dive straight into the rendering loop, sensor orientation->camera orientation, minimizing the latency and accuracy drift.

If I recall, Carmack's video showed him doing just that, for D3BFG--though it's of course easier when you have the source code for your game.

3DOF mouse-emulation x/y and directx-injected z would probably be a good first effort, though.

Direct memory injected roll is the only way you could implement it because there is no I/O device support for roll. It would be handy though to have the tracker poll be generic instead of tied to a specific device. Instead of polling the Hillcrest unit directly using their API, you could poll a pipe/socket/etc. The mouse emulation utility would then be put in charge of yaw, pitch, and roll which can be advantageous (code-wise), and also allow a variety of tracking devices to be utilized.

I thought Emerson is intercepting the render matrix so he can apply any transformation to it he wants, including roll, the game doesn't need an interface for this. It will go sensor->emerson's software->directx matrix transformation, or whatever. The only reason why modifying the camera in memory would be better is so that there isn't any culled geometry, but with roll it probably won't be much of an issue as you're still looking in the same direction (so the orientation of the frustum along the z-axis that the game uses to cull geometry won't be the same as the camera's orientation, but it's still looking along the same z-axis, so most geometry that needs to be shown will probably be shown).

That's basically accurate. Modifying the camera in memory might lower the overall latency, if the rendering loop runs at a faster rate than the game's input polling.

Roll is perfectly possible, in fact I just added it real quick to my driver right now:

Awesome

Do you know if you're going to get a RIFT slightly early Cyber? Or maybe get another go with the PR4? Would be a big win to get L4D2 RIFTracking at launch. Looks like you basically just need to get the warping nailed and the double rendering, then its just cleaning up your code and making a nice interface for different trackers. Great stuff.

BTW if you set the game to 4:3 aspect you should cut down on seeing culled stuff when you roll. TBH though nobody is going to roll more than about 15deg each way.

I'd be worried about running my driver against a Source game since I've made absolutely no effort to dodge VAC...

@Chriky: Not sure if I can get a pre-release Rift, but I will try to have Palmer test the driver as soon as I get something working good. Even if I get the kit the same time as everyone else, it would probably only add another week or two to get it optimized for the Rift and the tracker it comes with.

@Emerson: It should be fine as long as you don't play online.

Been working on compatibility a bit, the past few days. Made some progress!

Awesome work Emerson!

Hell yeah - this is the title I want the most! You the man Emerson.

Very cool, almost the full 9 yards now, but someones got to say it... "Can it run Crysis?"

The stereoscopic stuff is going to be problematic, but I'd love to try the Wiisis tracking mod with he Rift.

Any takers?

I hope so, I'd love to use this with Mechwarrior Living Legends!

I've gotten side-by-side mode mostly working. The main issue now is crashing sometimes when the game starts. First it was working only in windowed mode, now it only works in full-screen (though I didn't change any code). Not sure what the issue is, have to investigate further. But if you get in the game its fine. Also there are some small glitches with things like fire, but I'm sure I can fix it.

Very cool!

Watch on YouTube for the 3D options.

@cyber: Great work! Really impressed with the speed that you and Emerson were able to get to this point. At this rate we may have half a dozen Rift-ready games on release. Who needs Tridef?

(FYI: I ordered a Hillcrest tracker last Thursday and I hope to get support for it working within a week or two from now)

The Roll is also a very exciting prospect. If you have that much control I assume it might be possible to do variable height and some sort of head-translation as well. Head translation is a little weird though because essentially it's the same vectors as walking - just limited to a few centimeters around the primary "body" location. So for instance, something like peeking around a corner could be handled both by a direct memory translation matrix or by a mouse emulated strafe. I guess the difference would be that the translation matrix approach would not change the character position and bounding box in the simulation. Also - a lateral head motion on a "rolled" head would be a different motion. True "neck simulation" would certainly need to be handled carefully.