Cyber's DIY Stereo Driver [Work Log]

Post by **cybereality** » Sun Jun 24, 2012 12:45 pm

@PalmerTech: I still have some work left to do with the roll. It looks good in 2D, but in 3D its pretty bad since it messes up the stereo alignment (major headache inducing). I can probably sort it out somehow, just needs more work. I'm hoping to support the Hillcrest tracker used in the Kickstarter bundle as soon as I can get a hold of the hardware. Down the road I plan on making a more generic solution that will work with different hardware and software, but that probably won't be done for the Rift launch.

Post by **Likay** » Sun Jun 24, 2012 1:48 pm

Awesome work! I'm impressed how little time passed until a progress like this!

Post by **mm0zct** » Sun Jun 24, 2012 5:49 pm

Since you were talking about RTS games, Palmer, that's exactly what the game I was talking about adding support to is (kindof).

I know this is off topic but since the game does not involve and resource management (it's more of a real time tactics game) it would be possible to play the game without any of the HUD, so I'll probably be adding support for hiding various HUD elements when I add 3D support.

One of the things I am quite proud of in the game is the camera control, which will hopefully still feel fairly natural with the Rift and head tracking. (Black and white (1/2) would be a fun game to get working too).

Amazing progress with the stereo middleware, it's amazing how productive one can be in "bedroom hacker" mode.
If you want an Ogre application to fiddle with then let me know and I'll send you the game I was talking about. If anyone else cares about it it's Epoch Defence, easy enough to find on the Dare to be Digital website with google (we were a 2010 team). Currently it doesn't work properly in tridef or iz3d because of the terrain shader.

Post by **CyberVillain** » Fri Jun 29, 2012 5:20 am

cybereality wrote:@PalmerTech: I still have some work left to do with the roll. It looks good in 2D, but in 3D its pretty bad since it messes up the stereo alignment (major headache inducing). I can probably sort it out somehow, just needs more work. I'm hoping to support the Hillcrest tracker used in the Kickstarter bundle as soon as I can get a hold of the hardware. Down the road I plan on making a more generic solution that will work with different hardware and software, but that probably won't be done for the Rift launch.

No need to reinvent the wheel, FreePIE is around you know

Post by **cybereality** » Fri Jun 29, 2012 12:15 pm

@CyberVillain: Yes, I know. I should have been more clear. My idea is to expose a generic 3DOF tracking device, which can be accessed with plug-ins. Then anyone can add support for trackers themselves by compiling a plug-in for their specific device. So in order to support FreePIE someone (probably me) can create a FreePIE plug-in that will then support whatever FreePIE does. This could also work with VRPN or using device APIs directly.

Post by **CyberVillain** » Thu Jul 05, 2012 2:18 am

cybereality wrote:@CyberVillain: Yes, I know. I should have been more clear. My idea is to expose a generic 3DOF tracking device, which can be accessed with plug-ins. Then anyone can add support for trackers themselves by compiling a plug-in for their specific device. So in order to support FreePIE someone (probably me) can create a FreePIE plug-in that will then support whatever FreePIE does. This could also work with VRPN or using device APIs directly.

Missed this post, sounds perfect!

Post by **cybereality** » Thu Jul 05, 2012 9:06 pm

Got warping working thanks to Emerson.

L4D_Cyber3D2.jpg

Post by **brantlew** » Thu Jul 05, 2012 11:31 pm

Looks good. Question - so are you able to configure the FOV to match the Rift 90 degrees. Is that just a game setting or is that something that the post processor could do. I guess there might be issues with clipped geometry if you tried to do it post, right?

Post by **CyberVillain** » Fri Jul 06, 2012 1:03 am

Cool! Would it be possible to move the HUD further in? Out there it wont do much good

Could it be made generic or do you need custom code for each game? Could it be fixed with some kind of profile system?

Post by **cybereality** » Fri Jul 06, 2012 1:40 pm

@brantlew: Yes, I can control the FOV. However I have not figured out how to expand the frustum yet, so objects on the side get clipped. Its probably possible, though, just need to toy with it more.

@CyberVillain: Well I have not been able to move the HUD around yet, but there is probably a way to do it. Honestly what I want to do is just hide the HUD altogether, as I feel like it ruins immersion (UT2004 is still one of my favorite games in 3D because you can turn off the HUD/crosshair completely). So I can investigate this, don't know off-the-top how to do it.

I am looking for a generic solution, but there will likely still need to be profiles tailored to specific games (or at least specific engines). So far Left4Dead is the only game I really have working, but I am going to try to get at least a couple games supported before the Rift comes out. What I want to do eventually is have a extendable system, where there are a number or presets and options that could be set in an XML file on a per game basis. That way users could create their own profiles for new/obscure games. I doubt that will make it into the first release, but that's how I want it to work.

Post by **CyberVillain** » Fri Jul 06, 2012 2:29 pm

In Left4Dead the HUD is pretty important, if you remove it you cant tell if you have Medpacks, molotov etc... But yeah, in most games the HUD is not needed.
I really like Dead space one and two were the HUD is in 3D and part of the world

Post by **CyberVillain** » Sat Jul 07, 2012 2:57 am

cybereality wrote:Got warping working thanks to Emerson.

L4D_Cyber3D2.jpg

I tried this with the SBS feature of the HMZ, you probably knows this, but its something wrong with the Stereo, the shotgun is completely different for the two views

Post by **cybereality** » Sat Jul 07, 2012 1:36 pm

CyberVillain wrote: I tried this with the SBS feature of the HMZ, you probably knows this, but its something wrong with the Stereo, the shotgun is completely different for the two views

Can you be more specific about what you think is wrong? I didn't notice anything on the shotgun.

Post by **Likay** » Sat Jul 07, 2012 2:20 pm

I think it's the general problem with guns (gigantic gun, sometimes even "pointing through walls" if standing too close etc) and source engine. Iz3d (simple projection) and tridef have adjustments for those. It's not really a fault of the stereodriver itself.

Post by **cybereality** » Sat Jul 07, 2012 9:32 pm

Ok, I have tested with a few more games and either they crash at startup, or fail to inject properly (no crash, but nothing happens). The good news is that Portal 2 seems to work OK. There are some bugs with it, but it seems playable. Mainly the shadows are messed up (a common problem even with existing drivers) and also the camera separation seems to change slightly between levels, or even parts of the same level (not sure why this is). But at least its somewhat working.

P2_Cyber3D.jpg

Post by **CyberVillain** » Mon Jul 09, 2012 1:59 pm

cybereality wrote:
CyberVillain wrote: I tried this with the SBS feature of the HMZ, you probably knows this, but its something wrong with the Stereo, the shotgun is completely different for the two views
Can you be more specific about what you think is wrong? I didn't notice anything on the shotgun.

Im at the country house so It will take some time to test again, and get back to you...

Post by **cybereality** » Fri Jul 13, 2012 3:43 pm

[youtube]http://www.youtube.com/watch?v=VN8O8fA_5mg[/youtube]

Post by **brantlew** » Fri Jul 13, 2012 8:50 pm

Very, very cool. Works better than I imagined it would. How far do you have to roll before you really start noticing the clipping problems?

Alright Emerson - Cyber's thrown the gauntlet down. How about some roll support in your SkyRim driver? I can lend you a Hillcrest tracker and the interface code if that will help you out. I want that SkyRim mod bad.

Post by **CyberVillain** » Sat Jul 14, 2012 5:39 am

Nice work! Btw the work, I do not think there is anything wrong with your pic we discussed earlier, just to much seperation so I see the shotgun double. Same with the L4D clip above, way to much separation for my taste, so really nothing wrong...

Post by **rajveer** » Sat Jul 14, 2012 7:37 am

Looking awesome, can't wait to try it out! Didn't notice any clipping either, what I did notice however is that the right view has a yellow tint?

Post by **cybereality** » Sat Jul 14, 2012 11:58 am

CyberVillain wrote:I do not think there is anything wrong with your pic we discussed earlier, just to much seperation so I see the shotgun double. Same with the L4D clip above, way to much separation for my taste, so really nothing wrong...

Well I usually like to pump the 3D settings up so the environment has a nice amount of depth, but this usually makes the gun too hard to focus on. Otherwise what happens is the gun looks great and in 3D, but the background is pretty much flat (similar to some native games like Deus Ex).

Post by **CyberVillain** » Sat Jul 14, 2012 5:34 pm

cybereality wrote:
CyberVillain wrote:I do not think there is anything wrong with your pic we discussed earlier, just to much seperation so I see the shotgun double. Same with the L4D clip above, way to much separation for my taste, so really nothing wrong...
Well I usually like to pump the 3D settings up so the environment has a nice amount of depth, but this usually makes the gun too hard to focus on. Otherwise what happens is the gun looks great and in 3D, but the background is pretty much flat (similar to some native games like Deus Ex).

We have different tastes

Like the room you start in that clip (the very first room after the roof top in that level) Look at the room depth at your settings, its way too deep (its a motel living room), look at the UZI, its like a meter long(!)

Post by **Fredz** » Sun Jul 15, 2012 11:33 am

Shouldn't the S3D rendering be dependent only on the eyes separation ?

Post by **cybereality** » Sun Jul 15, 2012 11:54 am

Fredz wrote:Shouldn't the S3D rendering be dependent only on the eyes separation ?

If you are referring to the clipping/culling then no. The issue is that the aspect ratio of the screen is not square. So things on the top or bottom of the screen would normally be culled, but if you roll the camera 90 degrees then you are seeing more of the top and bottom of the view, and the game still thinks these areas are outside the frustum. In practice, its not a huge issue since you will rarely roll your head 90 degrees in real life (normally you will not go over 45 degrees). So its not a huge issue.

Post by **cybereality** » Sun Jul 15, 2012 6:36 pm

Now I've increased FOV to 90 degrees and also fixed the aspect ratio:

[youtube]http://www.youtube.com/watch?v=sDNOFeoN5Zc[/youtube]

Currently there are a few issues with this: objects getting prematurely culled, and the player names in strange positions. But it works good enough to play.

Post by **brantlew** » Sun Jul 15, 2012 7:49 pm

Looks good. What about the vertical FOV? Does it have to be increased to 110 to match the Rift as well?

Do you plan to make the Rift configuration optional? This would be useful simply as a 3D driver for any HMD - especially with the roll integration which differentiates it from other drivers like TriDef. I think the roll integration would be useful even for 2D mode.

What games do you plan to tackle next?

Post by **cybereality** » Sun Jul 15, 2012 9:30 pm

brantlew wrote:Looks good. What about the vertical FOV? Does it have to be increased to 110 to match the Rift as well?

I think its probably fine like this. If you look at one side of the screenshot the vertical view is actually a lot bigger than the horizontal (almost 50% larger) so this should match decent with the Rift. It won't be exact, but should be good enough. I mean, I played on the prototype just with IZ3D drivers, and even with the wrong aspect ratio and 45 degree FOV, it was playable. So I think this will be a lot better, even if it isn't 100% accurate.

brantlew wrote:Do you plan to make the Rift configuration optional? This would be useful simply as a 3D driver for any HMD - especially with the roll integration which differentiates it from other drivers like TriDef. I think the roll integration would be useful even for 2D mode.

Yes. There will be a config file to adjust basic parameters (like FOV, aspect ratio, default separation and convergence, enabling warp, etc.). Eventually I'd like to have a GUI for this but I don't think I can have that ready for the Rift launch. Down the line I'd like to have more generic support for other devices, but right now I am focusing on the Rift exclusively. Since its still a little buggy, I feel like the early adopters of the Rift will be more forgiving. If and when I have a more solid build I can expand the compatibility.

brantlew wrote:What games do you plan to tackle next?

Currently I'd like to find a way to increase the view frustum, so that objects do not disappear when you move your head. Its even worse than it looks in the video, after I started testing on different areas. Hopefully there is a way to do this through DirectX, but I'm afraid it may be something being done in the game logic before anything even gets into the DirectX pipeline. I'm still looking into this though.

In terms of other games, well Portal 2 seems to work. I imagine Portal 1 or Left4Dead 2 may also work (maybe just with some tweaking). I may want to try Deus Ex:HR, as that game seems really cool. Having problems with most of the titles I've tested with, so there may be something more fundamental that is wrong with my method. I'm open to suggestions if you (or anyone else) has a favorite game you'd like supported.

Post by **PalmerTech** » Tue Jul 17, 2012 12:39 am

Mirror's Edge seems like it would be good, since you don't need a weapon, and it has a minimal HUD. EDIT: I know Emerson is supporting it, but having roll would make a huge difference, in addition to having the field of view rendered better.

This driver is awesome, seems like it could be really successful as a third party 3D driver! I don't know if you are ready at this point, but if you want to commit to a name for the driver, I would love to mention it on Kickstarter when the Rift launches, if not in the initial video, then in an update shortly after launch. If you have a site up with some demos and a donation button, I am sure you would get interest.

Post by **cybereality** » Tue Jul 17, 2012 10:11 pm

Well Mirror's Edge is one of my favorite games, so I will look into this (haven't tried it yet, maybe it will magically work...).

I would like to make this into a more full featured product, but I feel like that time is very far away. Not sure if branding it at this point would help or hurt me. But I will keep that in mind.

Post by **brantlew** » Wed Jul 18, 2012 2:21 pm

cybereality wrote:I'm open to suggestions if you (or anyone else) has a favorite game you'd like supported.

Maybe something with a slower pace like Dear Esther?

Post by **cybereality** » Sun Jul 22, 2012 7:19 pm

DearEsther.jpg

Dear Esther looks to be mostly working. Right now the only major issue I see is with the wide FOV and clipping/culling. If I leave the FOV at 45 (or adjust it with game settings) then it should be OK.

Post by **brantlew** » Sun Jul 22, 2012 7:45 pm

Oh awesome! That's pretty cool. The Tridef drivers don't work very well for me with this game. It works fine until I look down at the ground and then the textures sort of break into a layered 2D pattern. You haven't seen any weirdness like that have you?

Post by **cybereality** » Sun Jul 22, 2012 9:20 pm

Yeah. If you look down then the floor disappears. I thought it was because of the FOV change. Will have to test more though.

Post by **brantlew** » Wed Jul 25, 2012 9:17 pm

Another idea for a game. I've been digging around the Valve website trying to figure out what's involved in using their SDK. As far as I can tell it's free to use for non-commercial modding (not 100% sure though). Anyway here are the list of games that are compatible with the SDK and I assume can be interfaced with the Head Tracking API.

https://developer.valvesoftware.com/wik ... Source_SDK

https://developer.valvesoftware.com/wiki/Head_Tracking

I haven't found the specifics about character motion yet, but I bet that could be modded as well to allow for analog speed/direction and variable height. Oh, and you can mod the camera FOV.

So potentially we're talking about a game mod + Rift correction for any of these Source games that would have the hooks in it for full VR motion control without any clipping problems. (Don't know about independent gun - but maybe that as well). That would be a really nice demonstration platform for testing full VR immersion. With the right trackers - it could be even more immersive than the Doom3 demo.

Post by **WiredEarp** » Thu Jul 26, 2012 12:30 am

Yep, the Source stuff looks pretty promising. I don't know how easy it will be to do separate gun movement however - since I assume most of the games use the head vector to aim the gun as well, you'd have to separate that somehow. Having said that, it could probably be hacked in, but I really hope that Doom 3 BFG supports independent aiming OOTB.

Post by **Alkapwn** » Thu Jul 26, 2012 9:22 am

I'm hoping this is the exact thing that Palmer and Carmack Have been meeting with Valve about. Because if they get all the parameters needed from those two guys, I'm sure it wouldn't be hard for them to add Rift compliance the code and SDK. To have those games as Rift Ready for the Rift launch would be a huge bonus to be able to show full games to play and demo the Rift.

Post by **cybereality** » Fri Jul 27, 2012 12:57 pm

@brantlew: The Source SDK stuff looks promising but, to be honest, not a whole lot of games use the Source engine. Currently I am trying to find a more generic solution that will work with more games, not just a couple. However, using the SDK would simplify a lot of things and could be more robust than trying to hack DirectX. And as much as I'd like to explore this, I only have so much time to dedicate to my driver before the Rift launches.

Post by **cybereality** » Sun Jul 29, 2012 5:15 pm

So I have spent the whole weekend completely re-writing the driver from scratch. While the old version did work somewhat, it had a lot of problems with instability with too many games. This new version is a lot cleaner and will be more easy to maintain (ie object-orientated). The old version was basically one huge Main file hack (LOL!!). So far I am still only supporting Source engine games, but it should be easier to add others (next up will be the Unreal Engine).

Anyway, now Half-Life 2 works!

HL2_3D.jpg

Post by **rajveer** » Sun Jul 29, 2012 6:44 pm

Awesome! How's the frustum clipping with the new version (does it still clip objects at the top and bottom)?

Post by **Fredz** » Sun Jul 29, 2012 7:31 pm

Looks very good so far, one of the most realistic shots you posted.

I found a free FPS game which has freelook support (True Combat: CQB), maybe it would be interesting to try to support it when headtrackers are available for the Rift.

From http://www.truecombatelite.com/index.ph ... qboverview :

"Freelook is probably the biggest and most unique feature in CQB. It's when your player model can look from side to side while keeping your weapon in the original forward position. Keep in mind that you can still shoot while in freelook and that the more you move to the side, the more your weapon will shift/sway."

Example of freelook in action :

Freelook is an optional bind in the game, so you can choose whether or not to use this feature.