PS3's new 3D mode coming in 2010 to all existing games

[metalqueen]

PS3's new 3D mode captured on video, coming in 2010 to all existing games
by Paul Miller

You know what's absolutely useless? A video of Wipeout HD being played in 3D, with some schmuck wearing 3D glasses and babbling on about how much fun he's having. Well, that schmuck is this Engadget editor, the video can be found after the break, and we've gotta say: we loved it.

Read the whole story here: http://www.engadget.com/2009/09/04/ps3s ... all-exist/" onclick="window.open(this.href);return false;

[DmitryKo]

PS3's new 3D mode captured on video, coming in 2010 to all existing games

Only if and when major publishers (or most likely Sony themselves) will go through an enourmous task of preparing a special game executable using S3D-enabled version of Sony's custom OpenGL ES and alter the visuals to match stereoscopic programming guidelines.

Think of Xbox 360 with its downloadable compatibility patches for original Xbox titles (which stopped coming at some points, though the coverage was still far from 100% - and that suggests it was primarily the effort of Microsoft engineers who probably had access to the source code and patched it for the Xbox 360, and that original game developers and publishers could care less).

The rest of the article is the usual techobabble with little regard to reality.

[ssiu]

If iZ3D can create an S3D driver for computer games (without being able to patch the games nor the ATI/NVIDIA graphics driver), then theoretically Sony should be able to do something similar for PS3 games. Performance (drop in framerate) is a big issue -- does the PS3 have enough hardware reserve to compensate? And it will be imperfect (just like the iZ3D / NVIDIA S3D drivers). But it is good news to hear that Sony is at least investigating it.

[DmitryKo]

If iZ3D can create an S3D driver for computer games (without being able to patch the games nor the ATI/NVIDIA graphics driver), then theoretically Sony should be able to do something similar for PS3 games. Performance (drop in framerate) is a big issue -- does the PS3 have enough hardware reserve to compensate?

It's not even about performance drop, it's about the way console games are made.

Gaming consoles are embedded devices and do not have user-installable "drivers" in the sense of Windows, MacOS, or Linux (in it's more user-friendly flawors, though it powers many embedded devices as well). Instead, the gaming console's "firmware" merely acts as a boot loader (a pretty cute and advanced one, though) and provides centralized setup.

All of the hardcore stuff in the current generation is done at the middleware level - in Sony's case, graphics are driven by a modified OpenGL ES library, which uses Cg as the shader language instead of standard GLSL. And the core difference between console middleware and user-installable drivers is dynamic linking versus static linking.


With user-installable "driver", the executable code resides in a dynamically-linked library which exports some set of low-level functions required by the OS. At DLL load time, the OS looks through the export table for these predefined names and gets their respective addresses within the DLL, then invokes them when necessary. Driver programmer's responsibility is to implement these functions by directly interfacing with the underlying hardware in a sensible manner. This scheme has several benefits: the OS can support any new hardware that has a driver, because the low-level implementaion details are maintained by the driver; driver bugs can be fixed independently from application and OS bugs by installing an the driver executable.

However, embedded devices often have fixed, non upgradeable hardware. So instead of providing a generic set of low-level functions than can suit many different devices, it's more efficient and easier to directly design software components around the specifics of the underlying hardware. So on the consoles, the low-level "driver" code dealing with fixed hardware is statically linked into the host application and game executable includes all the low-level middleware code necessary to operate the fixed hardware. There is no dynamical linking, and consequetntly no installable drivers and external modules. The self-contained game executable directly loads from a non-writable media such as game cartridge or optical disc and takes full control of the console (though current advanced consoles can replace the game executables with a newer version, either to fix bugs or to achive greater performance).

The low-level code is either provided by the manufacturer, or the developers have to write it all by themselves, using information provided in the Developer's Kit. And this is how things were since the very first Atari in 1970s to the Xbox and Playstation.


So no, older console games can not be updated to stereoscopy just by providing some updated "driver" or "library", like they do this on the PC or on the Mac. That's why Microsoft had to manualy rework the code of original Xbox games to work on Xbox 360, which uses entirely different hardware, and Apple could get away with radically changing hardware platform for the Mac several times (using some clever emulation software to run Motorola 68000 code on PowerPC processors, then to run PowerPC code on Intel processors).

[martinlandau]

DKO there was an entire panel discussion at PAX by MSFT XBOX employees and developers about all these various issues and they went into very long and technical discussions that confirm what you say. Changing and updating on the consoles is not going to happen like it does on a PC. This frustrated many of the audience members who wanted user community forum suggestions about updates and changes to be incorporated into the games, and they flatly said it is not going to happen. Just to make a few minor changes could take up to 6 months of testing for console games. However to give you some hope, after the panel was over I talked to some of the MSFT XBOX people and they told me to go see an exhibitor at PAX, http://www.slickentertainment.com" onclick="window.open(this.href);return false; about their stereo3D title Scrap Metal. So they are very aware of S3D and it was great they know about developers in S3D for thier hardware, I was impressed with these MSFT people. They were very passionate gamers. Another big theme put out by Adam Sessler and one of his panels with many top game developers was the enormous amount of time and money that goes into modern games. 10's to 100's of millions of dollars and 5 years of lead time in many cases. He also brought up this issue of updating and patching games and how in some cases it took an entire year for the game to be "patched" to what it should have been at release time and he named some specific titles that said frustrated him.

[UndeadD3vi1]

So they are going to lower the resolution the game runs at to do it? As thats the only way I can think of that would allow the PS3 to maintaine the same frame-rates?

Edit: Is this fake then?

[BlackShark]

It's the engadget editor who goes too enthusiastic.
Sony never announced it would provide stereo3d to all games.

[cybereality]

It's the engadget editor who goes too enthusiastic.
Sony never announced it would provide stereo3d to all games.

Did you watch the video? The Sony guy clearly says its a "software update that will allow *all* PS3 games to play like this" (at around 0:20).

[BlackShark]

Yes and that guy is precisely the engadget editor and not a Sony rep making an official announcement.
Once again, i do not know how sony plans to do this but i strongly believe that the engadget editor missed some of the info and suddently got over-enthusiastic about 3D because now he has in front of him a 3D product he actually wants real bad.

Remember what Blitz said about making 3D on a console : it is possible but hard. A simple quick and dirty firmware update won't make it magically happen like 3D drivers on PCs.

[cybereality]

Remember what Blitz said about making 3D on a console : it is possible but hard. A simple quick and dirty firmware update won't make it magically happen like 3D drivers on PCs.

Maybe not, but all this buzz is probably showing Sony that there is in fact demand for this product. So there could be a simple solution for adding stereo to PS3 games. It might not be backwards compatible, but I imagine Sony could release what they did on their developer network to allow any future games to easily support stereo 3d.

[DmitryKo]

Changing and updating on the consoles is not going to happen like it does on a PC.
Just to make a few minor changes could take up to 6 months of testing for console games ... in some cases it took an entire year for the game to be "patched"
Another big theme ... was the enormous amount of time and money that goes into modern games. 10's to 100's of millions of dollars and 5 years of lead time in many cases.

Thanks for this interesting commentary which adds up to the big picture. With no auto-magical path to stereo-3D conversion as on the PC, it is utterly unrealistic to expect game developers to go back and substantially re-engineer already released titles, when even small bug fixes take a substantial amount of time and money to test.

I imagine Sony could release what they did on their developer network to allow any future games to easily support stereo 3d.

Yes, but then the developers would have to go trough redesigning large portions of the game interface to make it stereo-compatible - things like object highlighting and selection, tags and labels, cursor and crosshairs, etc. are traditionally done via on-screen projection but need to be reimplemented to use depth of the object.

Its possible that what Sony is doing is similar to the Philips WoW and other auto-stereoscopic screens in that it is based on 2D+depth.
Also, since it was sending only an additional grayscale image (possibly at a reduced resolution) the bandwidth requirements would be much less than the full stereo pairs.

You can't save video bandwidth this way - no video interface allows switching resolution and color every frame. WOWvx packs 2D+Depth image in a single frame trasmitted at 1080p, and 2D and depth images are 960x540 pictures placed side-by-side, with extra blank lines (see 3D Display Interface White Paper). The only reason why WOWvx displays use 2D+depth is they are multi-view autostereoscopic and need to render 9 stereo views, and each of these views is only 640x360 pixels. A checkerboard-pattern two-view format would be more appropriate for two-view stereoscopic display with only a single stereo view.

I imagine it would be possible to gain access to the Z-buffer in the PS3's video memory via some running background process (aka 3D driver). The PS3 could then send this depth-map image to the TV which actually does the processing to produce two stereo pairs. This method would require next to no additional processing power on the PS3 end so resolution and framerates could be maintained.

Yes, in theory that would allow the PS3 to skip rendering two stereo frames and just send the contents Z- or W-buffer, but the resulting stereo picture will look like a cardboard miniature, and that visuals would only work for some of the simplest games.

Why would you think iZ3D and Nvidia stereo drivers go as far as auto-magically altering geometry and pixel shaders passed by the application to account for stereo separation between two views, and render the two views in two separate framebuffers each with its own Z-buffer?

Imagine a close-up of some object in the game, which pops out of the screen right in front of you, so the two stereo views of this object will be sufficiently different as each eye would be seeing a different side of this object. Do you realistically believe some magic chip would recreate such high-quality stereo image using just 2D+Depth information, when a proper stereoscopic renderer would need to take actual geometry of the object into account to deliver substantially different pictures? No way, it would look like a cardboard placed in front of the screen, nothing more.

To be fair, Philips tried to address these issues with further optimising the 2D+Depth format with two additional occlusion layers, called Declipse format, similar to Layered Depth Video (LDV) and Multiple Video-Plus-Depth (MVD) discussed at the EBU-ITU-SMPTE 3DTV Workshop. Suddenly it doesn't look all that simple, does it?

So they are going to lower the resolution the game runs at to do it? As thats the only way I can think of that would allow the PS3 to maintaine the same frame-rates?

Resolution is just a part of the equation. It's more about graphics effects, and some of them are too computation intensive even on today's PC parts, things like smoke, ambient shadows etc. "Fortunately", 4 years old GeForce 7800 arhitecture is not capable of realistically delivering most of them without taking a significant performance hit, so PS3 game developers have no option but to use simpler graphics.

[cybereality]

Yes, in theory that would allow the PS3 to skip rendering two stereo frames and just send the contents Z- or W-buffer, but the resulting stereo picture will look like a cardboard miniature, and that visuals would only work for some of the simplest games.

Yes, I agree that this is not an ideal solution but it is the only method that seems to make any sense given the news articles floating around. Either that or they were just demoing altered builds of the games and making up total B.S. which I could believe as well. Bottomline is that something just doesn't add up in this equation and after the initial excitement wore off I'm starting to feel duped.

[DmitryKo]

total B.S. which I could believe as well. Bottomline is that something just doesn't add up in this equation

For all we know, there are only three hard facts: displays from Sony and Panasonic will be full-frame 1080p60 per eye and will use 120Hz shuter glasses; PlayStation3 will stick to 1080p60 HMDI interface even with new revisions of the console; and the first PS3 stereo games use anaglyph and frame packing, either checkerboard pattern for shutter-glassed based TVS sets from Mistubishi and Samsung, or sub-field and interleaving for passive polarized screens. Everything else is speculation and is most likely a result of gross misinterpretaion by the media, so we'll have to wait for more official statements from Sony to come with precise technical details. And regardless of technical possiblility to retrofit older games for stereo, there are other factors that will weigh negatively on studio's decision whether to pursue this possibility. New PS3 games are far more likely to support stereoscopy right from the start.

[martinlandau]

Thanks for this interesting commentary which adds up to the big picture. With no auto-magical path to stereo-3D conversion as on the PC, it is utterly unrealistic to expect game developers to go back and substantially re-engineer already released titles, when even small bug fixes take a substantial amount of time and money to test.

If there was someway to leverage the hundreds of millions of fans and users to let them do this, we need s3d tools for the gaming community to make modifications. It is ENTIRELY impractical to expect a few handful of developers to do this, but say for one game you have 10,000 dedicated fans and many of them with programming experience who will do this passionately in thier freetime and you can get some things done. At the PAX conference there were MANY very high quality game developers who were unemployed because of the bad economy. Masters level degreed game developers with a decade or more of experience couldn't find jobs, they have lots of free time and all those skills could be applied to these kinds of problems - is sad to let them idle and very wasteful.

Resolution is just a part of the equation. It's more about graphics effects, and some of them are too computation intensive even on today's PC parts, things like smoke, ambient shadows etc. "Fortunately", 4 years old GeForce 7800 arhitecture is not capable of realistically delivering most of them without taking a significant performance hit, so PS3 game developers have no option but to use simpler graphics.

I have to hand it to Adam Sessler with G4TV, I have an article coming up about his and hal halpin's expectations of S3D gaming, but he specifically mentioned smoke and occlusions and a few other things as the real challenges and problems that he foresaw.

[mickeyjaw]

With respect to the problem of getting a 120hz 1080p 3d signal through a standard HDMI connection, is it not forseeable that maybe the HDMI link runs at 60hz (30hz/eye) and the post-processing in the TV is in fact a 3d compatible 60-120hz conversion. To achieve this, the PS3 would transmit a frame sequential signal, and the tv would buffer the last 2 frames recieved:

HDMI Signal: L1 R1 L2 R2 L3 R4 L4 R4
3d TV display: L1 R1 L1 R1 L2 R2 L2 R2 L3 R3 L3 R3 L4 R4

This technique would allow 120hz flicker-free display through a 60hz link at the expense of a delay of 3 120hz frames. It would also result in either a half frame update rate (30 full 3d views/sec) or the possibility of headaches during high speed horizontal movement, depending on how the device transmitting the signal behaves. I do not believe that reducing to 30hz 3d frame rate is a problem, as realD only show films at a 24hz 3d frame rate (but 144hz refresh) and nobody seems to complain. Also, most of the delay (2 frames at 120hz/1 frame at 60hz) is due to the need to finish reception of a full progressive frame before display. Most modern TVs probably do this anyway, so in effect there is only an additional 1 frame at 120hz / 0.5frames at 60hz delay (although feel free to correct me if i am wrong):

HDMI Signal: 1 2 3 4 5
Std. TV display: 1 2 3 4

It is also concievable that the TV could buffer 4 frames at a time. This would allow generation of intermediate frames on the TV:

HDMI Signal: L1 R1 L2 R2 L3 R4 L4 R4
TV display: L1.0 R1.0 L1.5 R1.5 L2.0 R2.0 L2.5 R2.5 L3.0 R3.0 L3.5 R3.5 L4.0 R4.0

This method would offer much less jitter in high motion scenes at the expense of a further 2 frames at 120hz / 1frame at 60hz delay, bringing the total delay up to
5 frames at 120hz / 2.5 frames at 60hz or and additonal delay of 1.5 60hz frames over conventional 2d gaming. This assumes that there is no appreciable delay time incurred by the need to calculate the intermediate frames on the TV.

All that needs to be done to achieve the first case is to modify the firmware on a standard 120hz upconverting TV, as these already have the 2 frame buffer needed. In fact, you could get away with removing a lot of the processing power present in these devices as there is no longer the need to calculate the intermediate frames.

For the second case, there would be a need to increase the frame buffer size so that it can hold 4 frames instead of 2 and there would be no need for any extra processing power over and above what the 120hz panels already do, as they already calculate between frames.

[BlackShark]

If there was someway to leverage the hundreds of millions of fans and users to let them do this, we need s3d tools for the gaming community to make modifications. It is ENTIRELY impractical to expect a few handful of developers to do this, but say for one game you have 10,000 dedicated fans and many of them with programming experience who will do this passionately in thier freetime and you can get some things done. At the PAX conference there were MANY very high quality game developers who were unemployed because of the bad economy. Masters level degreed game developers with a decade or more of experience couldn't find jobs, they have lots of free time and all those skills could be applied to these kinds of problems - is sad to let them idle and very wasteful.

LOL
This would equal open-sourcing the code. Even the most open-source friendly company (Id software) waits for the next engine generation before opening it's code to the public.

Sorry but this topic started with an engadget editor doing false assumptions and is going nowhere.

[cybereality]

...we need s3d tools for the gaming community to make modifications. It is ENTIRELY impractical to expect a few handful of developers to do this, but say for one game you have 10,000 dedicated fans and many of them with programming experience who will do this passionately in thier freetime and you can get some things done.

No, not at all. For an experienced graphics programmer adding S3D support to an engine you wrote is pretty simple. I think that guy from Crytek said it only took him a couple of days. And there is no way a major developer is going to put the source-code to their latest engine up on the internet for free. You could just forget about that right here and now.

[smoothy]

Okay I just read through all that, So whatever method Sony are working on it's safe to say you need a Sony 3d TV. This is strange as Sony officially said that they were going to work with TV manufacturers to get their TV's working with their content from bluray players, cameras, and PS3. Which makes me think that it's not the TV doing the work it's the PS3.

[martinlandau]

...we need s3d tools for the gaming community to make modifications. It is ENTIRELY impractical to expect a few handful of developers to do this, but say for one game you have 10,000 dedicated fans and many of them with programming experience who will do this passionately in thier freetime and you can get some things done.

No, not at all. For an experienced graphics programmer adding S3D support to an engine you wrote is pretty simple. I think that guy from Crytek said it only took him a couple of days. And there is no way a major developer is going to put the source-code to their latest engine up on the internet for free. You could just forget about that right here and now.

I don't forget Cyber, you of all visionary types know the world is changing at warp speed, and the demands of 7 billion human beings going forward will NOT be controlled, it is totally out of control, I have seen it too many times with corporations in the past. That guy from crytek cannot go in with a few days and build 100 new levels, characters, storylines, etc etc properly in stereo3D like 1000 dedicated fans that are unemployed games developers in Seattle Washington. IBM said no one could do a PC without them. Years laters MSFT said no one could do an OS without them (linux). Years later Verizon said no one could do a successful phone launch without them (iphone) ETC ETC ETC - You may be CERTAIN of the future cyberreality, and of what developers will or won't do and what 7 billion humans will or won't demand, but the only constant I am CERTAIN of is CHANGE, and change faster than any one human can predict.

[DmitryKo]

All that needs to be done to achieve the first case is to modify the firmware on a standard 120hz upconverting TV, as these already have the 2 frame buffer needed

That's interesting, but what is fine for pre-recorded 24 Hz material will not necessarily do any good for gaming. Do you know that some advanced TV sets feature a special "game mode" which disables each and every image enhancement feature to reduce lag? Do you know that some advanced FPS gamers make a difference on an overdriven TN+film display comparing to a slower IPS/PVA display? I don't think you can get away with all this additional processing.

For an experienced graphics programmer adding S3D support to an engine you wrote is pretty simple. I think that guy from Crytek said it only took him a couple of days.

To quote a Murphy's Law, "If everything seems to be going well, you have obviously overlooked something."

That guy from crytek cannot go in with a few days and build 100 new levels, characters, storylines, etc etc properly in stereo3D

Yes, an often overlooked fact is that artistic assets like level and character geometry need to be verified for stereo3D.

Also, there are lots of common graphics effects which do not translate well into stereoscopy, because they are rendered in screen coordinates or triangle quads instead of actual geometry. This includes interface items like selection box and object highlighting, head-up displays, and object labels; particle effects like smoke and dust; billboard objects like trees and grass etc. Pixel shaders which do post-processing - things like blur and out-of-focus; mirrors, shadows and other effects that use render-to-texture - they all can be problematic as well.

It's just a brief compilation of issues listed in the MTBS3D Stereoscopic Programming Guide and Nvidia stereo presentations.

say for one game you have 10,000 dedicated fans and many of them with programming experience who will do this passionately in thier freetime and you can get some things done.

Well, one John Carmack could probably easily make for 10 000 "fans with programming experience". This is the case where quantity never makes up for quality.

he specifically mentioned smoke and occlusions and a few other things as the real challenges and problems that he foresaw

Was it just an observation, or maybe he has some solutions to offer?

MANY very high quality game developers who were unemployed because of the bad economy. Masters level degreed game developers with a decade or more of experience couldn't find jobs, they have lots of free time and all those skills could be applied to these kinds of problems - is sad to let them idle and very wasteful.

You're basically assuming that these developers will gladly work for food... oh, sorry, for free?!. Khm. OK. And then goes the best part: you just need to convince game studios to let a bunch of total strangers access their source code and other proprietary assets and submit uchecked modifications. 8-O Well, I don't really think it's ever going to happen, sorry.

[cybereality]

So whatever method Sony are working on it's safe to say you need a Sony 3d TV.

If thats the case then things are really going to get ugly. Maybe Sony plans to license their 3D tech to other companies, I don't know. But I can't see how Sony expects to support a whole market with just their TVs alone. So you could have people that already have a 3D solution for the PC, people that bought a Panasonic 3DTV for Avatar and other BluRays and then the Sony Bravia to play PS3 games in 3D? Thats not going to fly.

[smoothy]

we haven't addressed yet whether this PS3 3d tech is going to work for 3d bluray movies on PS3. From the linking I did the guy on cnet said the tech is not possible for movies.

[PalmerTech]

The moral of the story is that Engadget writers suck.

One time, a person submitted my gameboy backlight mod and claimed they had made it, and they did not credit me. Fine by me, but despite several emails and comments asking for a correction, I got completely ignored.

I would be buying a PS3 so fast if it supported 3D for all games past, present, and future, but I do not see that happening.