I'm adding native stereo to AssaultCube

[CarlKenner]

I'm adding native stereoscopic support to AssaultCube.

What does it need in order to pass certification, and what do people think constitutes good stereo in a game?

It's mostly for a VR HMD, so I'm mainly aiming for realism. But I don't know if that's what people like.

And how do I handle the weapon? It is going to be very close to your eyes compared to the rest of the world, and should be at least 2 metres in front of the screen for realism (the VR920 screen is supposedly 9 feet away).

And how do I handle the HUD?

I am only drawing the crosshair on the right eye, and I am making the right eye basically in the middle and the left eye an interocular distance to the left of the middle.

When using the sniper scope, I'm rendering the sniper scope on the right eye and the left eye shows the world as normal (unmagnified) plus the left side of the gun, just like it would in real life. So people will usually close their left eye while using the scope, like they would in real life.

I've also added an (optional) iron sights button for the other weapons, which are lined up for the right eye. Iron sights still looks OK if you have both eyes open, but some people might close their eye for that too. Using iron sights doesn't offer any advantage over the normal view, I just thought it looked cool and more realistic.

[crim3]

That's exactly how aiming should be done in S-3D!

The FOV should be configurable, so the FOV's HMD can be matched to avoid the scale model effect that gives S-3D with relatively in-game big FOVs.

Could be the HUD be drawed at a certain depth (several "in game meters") instead of screen depth? This would allow not to see a doubled HUD when looking at average distances. Though I'm not sure if that's convenient, should be tested.

What about independent head movement using head tracking?

[CarlKenner]

Yes, I made it so you can drop the FOV to as low as you want. Originally it's minimum was 60. But I figured people might want more realism, and my HMD is only 25.6 degrees (the game uses horizontal FOV). The default FOV was 100, which I think is unbearable on a HMD with a narrow actual FOV. Using the real FOV of 25.6 degrees is also not very good. But people can adjust it to somewhere in between.

But I don't think it is just the FOV that causes the scale model effect. You need to have the right convergence and separation. I used the real scientific values that they should be, so it doesn't look like a scale model.

Drawing the HUD at a certain depth is a bit tricky because it is rendered in 2D and has no concept of depth. I can try adding depth to it, but it is more work.

I haven't got independent head movement working yet. Independent head movement is tricky. Currently you have to aim with your head, which means you will be mince-meat when you are playing against those cheaters who use a mouse.

I did have a go at trying to control the gun seperately using a Wiimote, but I couldn't get it to work yet.

[CarlKenner]

Here is my native stereo patch for AssaultCube:

Edit: Here's the newer version. The stereo mode menu is now nested and has more options.

http://rapidshare.com/files/93508434/As ... eVR002.zip

You need to download AssaultCube from here (under 20 MB):

http://assault.cubers.net/download.html

Extract the patch into your AssaultCube directory, replacing your existing files. (should work now)

[pixel67]

Wow, nice work Carl. What formats are supported? If DLP3D is in the works then I can take it for a test spin.

PiXeL

[CarlKenner]

Is that the checkerboard thing?

[pixel67]

Yes, it is. There is a registry patch for the Nvidia drivers floating around that adds this support to 7900 cards, and the TriDef drivers have this profile as well. More info can be found here:

http://www.dlp3d.com/hdtv/3-d_dlp_hdtv.aspx#3d

Several links from that site including whitepapers and links to the actual mask for 1080p and 720p.

PiXeL

[pixel67]

For those that haven't checked this out yet, you are in for a real treat. Carl has added 11 (eleven) display formats to this game. NATIVE! I sure hope the guys over at GDC are paying attention here! All these developers out there getting paid 6 digit salary's, yet Carl added this in no time. Just a quick note on the installation. Do not autoextract the files into Assaultcube folder. Extract them to a separate directory and copy/paste them into the correct folders as the directory tree in the zip file is misnamed causing it to create a new folder rather than replace existing files. I have tested the anaglyph modes (plural!) and it works very well!

Neil/Sharky,
Might want to move this thread under a new section like Unigine as I believe this is a keeper!

Congrats again Carl on some very nice work. I will be interested in your feedback about adding DLP3D and possibly IZ3D support to the mix...

PiXeL

[CarlKenner]

Actually I added 19. I just thought that was too many for the menu. You can choose the other formats from the console by typing something like:

stereo3d 6

for example. I think 6 is magenta/green anaglyph mode. Which is why I didn't put it in the menu. By the way, why does nobody use magenta/green anaglyph?? It would be the optimal colour choice for making both eyes the same brightness. Red+blue combined are about the same brightness as green.

The supported modes are:

• Mono, which I think has a bug causing a hall of mirrors effect sometimes, but I can fix that.

• VR920 native stereo (working)

• Anaglyph (every possible colour permutation) (working)

• OpenGL's quad buffered stereo for Quadro and other similar cards. (untested, requires restarting the game)

• Page flipping (untested), as there is no way of swapping the eyes yet, this isn't much use.

• Blue line code, and white line code, frame sequential but not interlaced. Probably useless.(untested)

• Nvidia stereo driver mode in theory turns on and off the nvidia stereo driver automatically. But I have Vista, and I can't find the documentation for this API, so it may not work. Unlike all the other modes, the stereo is not handled by my program.(untested)

• Nvidia StereoBLT mode uses the nvidia stereo driver to write to the left and right eyes individually. This is unimplemented and may be impossible since I am using OpenGL.(unimplemented)

I plan on adding the following modes:

• Dual monitors in span mode, or single monitor side by side mode (same code does both). This should work on the TDVisor, but only if they don't have Vista (no span mode on Vista).

• Over under.

• Line interlaced. (shutter glasses mainly)

• Checkerboard. (3D DLP)

• Mirrored version of span mode

I don't know if it is possible to add iz3D support, because it is an OpenGL game. Also you have to sign an NDA to get them to send you their algorithm, which I'm not sure I want to do.

I'd like to add proper dual monitor support, but I don't really know how to do that in OpenGL.

By the way, many of the stereo modes it supports will still work in windowed mode.

[LukePC1]

Just a quick note on the installation. Do not autoextract the files into Assaultcube folder. Extract them to a separate directory and copy/paste them into the correct folders as the directory tree in the zip file is misnamed causing it to create a new folder rather than replace existing files.

I tried to get it to run, but my PC keeps crashing - even after I tried to follow your note.

I wonder, if I have to choose special options with the nVidia driver --> e.g. not S-3D on hotkey (like it is in the moment), but completly of...

[CarlKenner]

It is randomly hanging on startup for me sometimes. I have to Ctrl+Alt+Del it and run it again. Then it works.

Does AssaultCube crash without my patch?

Can you be more specific?

[CarlKenner]

I just got line interlaced mode working. So it should just be a few minutes to get checkerboard and column interlaced modes working.

[LukePC1]

Well I just tried again - without the patch. The result is the same, though

I get the loading screen, but that's all. It stays visible for some minutes, then I lost patience and truned off the PC. crtl+alt+del didn't work as well. This occured all 3 times I tried it.

That's all the information I can give you. If you find an other mistake of me, I'll try it again, but I don't want to restart my pc all the time

[pixel67]

Are your other drivers disabled/enabled (IZ3D/NVidia)? This may make a difference. I didn't have any problems starting the game. Changing the stereo profiles occurred in real time as well along with a decent menu for selecting separation, etc...

[pixel67]

I don't know if it is possible to add iz3D support, because it is an OpenGL game. Also you have to sign an NDA to get them to send you their algorithm, which I'm not sure I want to do.

Very good question, but this is also an open source game, right? No one is making money off of it so maybe they would be interested in helping as a PR/Research project. This is what we all want developers to do in the first place. I should have some time tonight to test DLP if you are able to get it working as well.

PiXeL

[CarlKenner]

I reckon I got DLP working. And interlace, vertical interlace, dual monitors in span mode/side by side, planar mirror, and above/below.
I just have to package it.

[CarlKenner]

Here's the newer version. The stereo mode menu is now nested and has more options.

http://rapidshare.com/files/93508434/As ... eVR002.zip

[pixel67]

You "reckoned" correctly! (are you from the south as well) DLP3D works like a champ! I have to say that I admire your programming abilities. Once I figured out how to change resolution settings in the .bat file, I was all set! Ghost free HD Stereo and not a third party driver in sight. Congrats on creating the worlds first game with native DLP3D Support! Lets see, it took you less than a day to add this in. Software developers....take note of this!

Thanks for spending your time on this Carl. I owe you a cold one!

[CarlKenner]

Yes, I'm from the state of South Australia, in the great southern land.

Sorry about not being able to change the resolution in-game. It's a Windows SDL thing. And unfortunately the game is written so that all the initialisation code has to be done in order, with the graphics initialised before anything else (including before loading the saved settings).

I added a -q command line option that is needed if you might want to use quad buffered OpenGL stereo (only works on Quadro and similar cards). That option doesn't activate quad buffered stereo, it just allows it to work if you choose it in the menu.

There is also a -s command line option that stops DLP and interlaced support from working, but may (probably won't) save a tiny amount of a video memory in other modes by not creating a stencil buffer. The original game didn't create a stencil buffer.

I think you can also run it in a window with -t. But you will have to make sure you put the window at even numbered pixels. It doesn't read the window position yet to adjust the stereo mask, so there is a 50% chance of the stereo being backwards when you move the window.

By the way, to use under/over stereo mode properly, you probably need to set the "blankband" console variable to the number of scanlines of blanking needed in the middle of the screen.

I just realised that the dual screen span mode and the side by side mode need to be separated into two different modes, because they should use different aspect ratios. Currently on a TDVisor, everything would look stretched vertically. Oops. I'll fix that later.

[The_Doctor]

Unfortunately games that sell well take millions of dollars and big corporations are all about money. Not spending and extra cent to add anything "useless" as they probably call it. Read this:
http://ea-spouse.livejournal.com/274.html . I know small studios are better since a friend of mine works for one, but usually you are still at the mercy of the publishers and such. Gone are the days when one or two people could come out with a super-game.

[pixel67]

That is deplorable and another great reason why opensource may be a good way to kick off initiatives such as this. Ground zero support with people working on projects because they love what they do, not because a billion dollar corporation is forcing them to do it...

[CarlKenner]

I'm having trouble adding Sharp3D laptop and LCD support. I think there is a serious bug in my stereo rendering.
In anaglyph modes the gun in the left eye is in a different position than it is in interlaced (and checkerboard modes).

Sharp3D is subpixel interlaced, meaning I need to render the scene 4 times, twice for each eye. The problem is it is rendering the two left eye views in different locations, when it should be rendering them in exactly the same location. The gun is different for the two left eye views. I think it is due to the same difference between anaglyph and interlaced modes.

EDIT: OK! I figured it out! the rendering function reads back from the depth buffer, and uses that to find the world coordinates of the point in your crosshair. Unfortunately the weapon drawing code was using a line between you and that point to work out which direction you were facing, and position the gun accordingly. The crosshair is only lined up correctly on the right eye, for the left eye that point is rubbish. In anaglyph and flip modes it reads that point from the left eye. In interlaced modes that point can never be for the left eye because those coordinates are for a pixel which only exists in the right eye.

So I'm going to stop rendering the gun based on that line between you and your target, and instead render the gun based on the euler angles. I'm also going to only read the distance from the right eye.

[CarlKenner]

That's exactly how aiming should be done in S-3D!
...
What about independent head movement using head tracking?

OK, I've got independent head movement sort of working. Actually it is independent gun movement since your head and your legs are locked to the same direction, at least in terms of yaw.

Basically your gun and your head now each have their own sets of yaw, pitch, and roll values. The gun crosshair is now displayed wherever the gun is pointing instead of locked in the centre of your view. And the gun model is rotated correctly. You can aim the gun at a specific point in the world and then turn your head as much as you like and the crosshair will stay locked on that precise point no matter how you turn your head. And surprisingly iron sights still works even when you aim a different place than you look.

All that is only possible because it is only 3DOF, and because only the left eye (which has no crosshair) is off-centre. If either the gun or the head were 6DOF tracked, the whole concept of a crosshair would make no sense, because it would have to be at a different point for every depth.

I've also added parallel and crosseyed naked eye stereo modes. It draws two narrow (1/6th the width of the screen) strips down the middle of your monitor, and you just look through the monitor (parallel) or at a point in front of the monitor (crosseyed) to see it in 3D. So for those of you who can't afford red/blue cardboard glasses, you too can see in 3D. I can't do crosseyed viewing so the crosseyed one is untested, but parallel works well.

And I have the new Sharp3D mode working perfectly (in theory).

I've just got to fix up some things for a release.

[crim3]

But I don't think it is just the FOV that causes the scale model effect. You need to have the right convergence and separation. I used the real scientific values that they should be, so it doesn't look like a scale model.

Thanks to your comment about convergence I have realized that I was using too high values for convergence. The lasts days I've been readjusting convergence and everything now looks a lot better, specially big cockpits in flight sims. I can see the real volume of the inner space of these ones. And also, it's a lot more comfortable for the eyes.

I think I was using better convergence values some months ago, and I've been incrementing them with the only purpose of achieving a stronger 3D effect. So it was my ignorance . Bad done! But now I have 're-rediscover' my sims and games.

I still haven't tried AssaultCube. I'm so hook up to flight sims that I almost don't do anything else than flying once I sit in front of my computer. That's what it happens when you have at your hands what has been a 15 years awaited dream.

[CarlKenner]

You are right about FOV though. Setting the other parameters helps a bit, but FOV is very important to making it feel the right size.

[shonofear]

Hey CarlKenner,
Im a fellow Ozzie and have been watching this thread and am pretty impressed with what youve archived.
I was wondering is your background in computer science (programing etc), if so, wats the basic needed knowledge to do wat you have done in making a game work in Stereo natively...?
is it all C++ with OpenGL coding...?

Keep up the good work matey,
I also tried the Assault Cube with me Shutter glasses (Nvidia driver, ver91.xx) but got artifacts,
ill have to try it again though, really interested

Cheers,

Shono

[CarlKenner]

If you mean the nvidia stereo driver, then yes, I got artifacts too when I tried it on my old computer with a similar version. The nvidia stereo mode isn't really working. It might be because AssaultCube reads from the depth buffer.

But the main point of this was to NOT use the nvidia stereo driver.

If your shutterglasses support line-interlaced, over-under, blue line code, white line code, or page flipping format, then you should try disabling the nvidia stereo driver, and choosing the correct stereo mode in the AssaultCube options menu.

You might need to use an "activator" program to put the glasses into that stereo mode.

If you are using over-under format, you will also need to set a console variable called "blankband". Press the console key then type "blankband 10" and press enter to change it to ten scanlines for example. I have no idea what the correct value should be.

It is all C++ with OpenGL coding. Although it usually doesn't use any object oriented C++ features, just normal C code. Open Source games tend to be OpenGL. They may also use SDL.

You need to understand the following OpenGL functions to add native stereo to games:
glFrustum (for setting the projection matrix)
gluPerspective (which you need to replace with glFrustum)
glColorMask (for anaglyph or subpixel interlaced)
glDrawBuffer (for quad buffered stereo on quadro cards only)
glStencilFunc, glStencilOp (for all interlaced modes)
glViewport (for side-by-side, over-under, dual monitor span, planar mirror, parallel and crosseyed modes)

For the Vuzix VR920 you need to check out http://forums.vr920.com for the code you need.

To do stereoscopic rendering, you need to render the scene twice*. It is not enough to just move the camera to the position of each eye, because then you would need two seperate screens with the centre of each screen over the centre of each eye. Even in HMDs that is not the case. You always have one (real or virtual) screen with the screen centre between your two eyes. Normally when you render an image, you render the centre of the image in the centre of the screen. We need to use off-centre (aka off-axis) rendering to render the centre of the image off to the side of the screen. That is what glFrustum is for. You need to use glFrustum AND move the camera in order to render for each eye.

* Sometimes three or four times for SubPixel interlaced to get the subpixels and the interlacing working together, but you render from the same position.

[Welder]

Ahh, it's OpenGL, ok then.
Would be cool to see it on our iZ3D Monitor.

Does the engine support QuadBuffer Mode by chance?

We have an alpha version of our Drivers for QB, so it may work.

--Charlie
Technical Support Specialist
iZ3D

[shonofear]

You might need to use an "activator" program to put the glasses into that stereo mode

please explain.... i have the cheap X3D Extreme Shutter glasses, not sure wat other modes it can display in.
It does Page Flipping with the Nvidia Driver.

Cheers

[Tril]

The E-D activator can do that. You choose Page Flip, LCD Mode Off and Stereo - On/Resync. The problem with this is that the eyes might switch often.

E-D Activator (http://edimensional.com/support_updates.php)

[shonofear]

cheers 4 that

[CarlKenner]

Ahh, it's OpenGL, ok then.
Would be cool to see it on our iZ3D Monitor.

Does the engine support QuadBuffer Mode by chance?

We have an alpha version of our Drivers for QB, so it may work.

--Charlie
Technical Support Specialist
iZ3D

You can see it on your iZ3D monitor... if you get a pair of anaglyph glasses and put the monitor in 2D mode But I think that would be considered traitorous
Or if you don't want crosstalk ghosting, I have a version here with parallel and crosseyed modes.

It does support QuadBuffer mode if you add "-q" to the command line in the .bat file and then set the stereo mode to quad buffered in the menu.

There is a bit of a problem adding iZ3D support to open source games. If I add iZ3D support I can't release the source code. For AssaultCube that would probably be OK in terms of its zlib license, but I couldn't add support to other games.

[Welder]

I'll try it shortly then

Games do not have to be tailored to work with our monitor, so since it has QB support it may work outa the box.

One thing though:

you say to add Q to the BAT, and then in game change it in the menu.

This could cause an issue because our Driver injects at the startup of the application.

Does Q set it to QB at start?

[CarlKenner]

That shouldn't be a problem. If you choose quadbuffer from the stereo menu it will start with quadbuffer the next time you run it.
With quadbuffered mode you need to enable it when the window is first created, which is what the -q option does. So that should be enough for your driver. If people choose a different stereo mode it is just wasting the extra buffers.

Does your driver still work if people don't have a quadbuffered graphics card? Most people don't have a quadbuffered graphics card. I know I don't.

[CarlKenner]

I'm adding support for Chromatek's Chromadepth 3D glasses. These glasses are not coloured, but they are prisms. So different wavelengths of light appear at different depths. Unfortunately a computer screen can only produce 3 different wavelengths because it only has 3 colours of subpixels. So close things are rendered red, screen level things are rendered green, and far things are rendered blue. I don't have any chromadepth glasses to test it with though. This should be the worst quality stereo effect, since it only provides 3 flat layers, but I feel I should add it for completeness.

[nubie]

Just tried it on a Planar setup.

It is OK, I find that no matter what FOV (32-180) it always has the wrong aspect ratio, looking up at a building and turning causes the building to change shape.

I tried 1280x960 mode and it presented me with a black bar across the top in both screens, and the problem didn't seem to go away.

Am I missing something? Is there a Square pixels mode? Or an Aspect setting in the config file/command-line I can pass?

Nevermind, Carl answered me, there is a newer version:

http://rapidshare.com/files/97540300/As ... Zalman.rar much better.

[nubie]

Is the head tracking support for trackIR implemented? Or is that for one of the HMD displays?

I got FreeTrack up and running, it is pretty nice actually. It seems to use the TrackIR method of DirectX interface, I don't know how the game reads it though.

[Ezzelin]

So I have an x3d set of glasses that don't activate when I select page-flipped inside of the game. Is there any kind of activator software for the x3d glasses? I'd love to try the native 3d stereo.

[yuriythebest]

ok the game has good 3d with anaglyph however- how do I tell the teams apart??? there seem to be green and blue however each time I shoot either one I kill a teammate??? very confusing....
also any way to adjust the color saturation?? the game is too bleak as it is even without anaglyph...