Michael Abrash talks about a number of problems facing VR/AR in this talk at this years GDC http://www.roadtovr.com/2013/03/31/gdc- ... lides-4415 . He describes the artifacts that arise in attempting to create an immersive VR experience. A main sticking point he had is the refresh rate of your stereoscopic display. That is of course driven both by the content being displayed and the processing power you have to drive the screen.
My idea for AR, no idea of it's possible, is to increase the frame rate in the areas on your HMD that is overlaying whatever info/object you have in your FOV.
For example, if you wore a HMD with stereoscopic cameras on the front of it that displayed its feed on the 7" screen inside it and you wanted to overlay a virtual apple on the desk in front of you, could you increase the refresh rate of the screen only surrounding the apple?
My questions:
1 - Are displays even capable of doing that?
2 - How would that affect the "natural" look of the scene? Would the faster FPS area look...off?
3 - Are there savings to be found for the GPU doing this, and is it even worth the trouble?
Localized FPS changes to alleviate strain on the GPU for AR
-
cheese_stick_mafia
- One Eyed Hopeful
- Posts: 1
- Joined: Wed Apr 03, 2013 9:07 am
-
Mystify
- Certif-Eyed!
- Posts: 645
- Joined: Fri Jan 11, 2013 5:10 pm
Re: Localized FPS changes to alleviate strain on the GPU for
I don't claim to be an expert in this, but...
I don't see any fundamental reason you couldn't do this. You could have your render cycle update a portion of the graphics buffer and let everything else persist. It would probably look weird and cause tearing if the entire scene is rendered, however. I can see this being useful if your screen is transparent, and you block out pixels to render virtual objects into the real world; the transparent pixels would not be rendered, and tearing would not be an issue.
I don't see any fundamental reason you couldn't do this. You could have your render cycle update a portion of the graphics buffer and let everything else persist. It would probably look weird and cause tearing if the entire scene is rendered, however. I can see this being useful if your screen is transparent, and you block out pixels to render virtual objects into the real world; the transparent pixels would not be rendered, and tearing would not be an issue.
-
nanicoar
- Cross Eyed!
- Posts: 101
- Joined: Mon Sep 10, 2012 10:09 am
- Location: Finland
Re: Localized FPS changes to alleviate strain on the GPU for
I'm an expert on everything, ever! =D
You could have something like parallax scrolling going on in the backdrop;
And render your complex, high-FPS stuff on top of it. You would be running at the maximum frame rate all the time since the hard limit on the frame rate comes from the RAMDAC, which hasn't really needed any improvement since the end of the CRT-era.
When the stuff you were putting a lot of GPU resources on no longer needs to be updated you would simply add its 2D image on a transparent background to your stack as illustrated in the above figure.
With pre-computed animations and parallax scrolling you could make e.g. the view from an in-game window look photo-realistic, provided you have a lot of RAM on your GPU. However if you managed to clip through the wall of the window and look at the art up close you would see that they are just flat coulisse; unless as you lucidly pointed out, the objects nearest to you could be rendered live.
Since GPU-type RAM comes at a premium a more popular approach comes from LOD-ing, where you create a single big, medium-resolution mesh for everything in the background. Then the really far-off stuff, e.g. distant trees are duplicated from a few 2D 'sprite' images into forests. This provides for the kind of performance/quality compromise we are used to seeing in games like Skyrim.
Perhaps MIP-mapped/JPEG2000 video with alpha channels, pre-emptively streamed into the GPU from a fast SSD, could bring down the RAM requirements to match a hobbyist's budget. - Just last month I was actually working on the streaming part of such code, but for web browsers and static images. I was able to demonstrate the most basic functionality before I was distracted from the project, which is now waiting in a repo on github (handle: ganzuul). If I don't make it happen soon, someone else certainly will.
You could have something like parallax scrolling going on in the backdrop;
And render your complex, high-FPS stuff on top of it. You would be running at the maximum frame rate all the time since the hard limit on the frame rate comes from the RAMDAC, which hasn't really needed any improvement since the end of the CRT-era.
When the stuff you were putting a lot of GPU resources on no longer needs to be updated you would simply add its 2D image on a transparent background to your stack as illustrated in the above figure.
With pre-computed animations and parallax scrolling you could make e.g. the view from an in-game window look photo-realistic, provided you have a lot of RAM on your GPU. However if you managed to clip through the wall of the window and look at the art up close you would see that they are just flat coulisse; unless as you lucidly pointed out, the objects nearest to you could be rendered live.
Since GPU-type RAM comes at a premium a more popular approach comes from LOD-ing, where you create a single big, medium-resolution mesh for everything in the background. Then the really far-off stuff, e.g. distant trees are duplicated from a few 2D 'sprite' images into forests. This provides for the kind of performance/quality compromise we are used to seeing in games like Skyrim.
Perhaps MIP-mapped/JPEG2000 video with alpha channels, pre-emptively streamed into the GPU from a fast SSD, could bring down the RAM requirements to match a hobbyist's budget. - Just last month I was actually working on the streaming part of such code, but for web browsers and static images. I was able to demonstrate the most basic functionality before I was distracted from the project, which is now waiting in a repo on github (handle: ganzuul). If I don't make it happen soon, someone else certainly will.