Photorealistic VR (Path Tracer)

[colocolo]

I thought i post something about a blog concerning real time path traced graphics for games.
I found it one year ago. a developer from OTOY is currently working on photorealistic graphics for games, the brigade engine 2. in his comments on youtube and his blog he claims that within a year this would be possible.
He says that there only have to be some improvements on the hardware side and software side. If you read some of his posts, u will understand why he is so excited. And me too, it could mean that u could be in a photorealistic (cinema quality) virtual reality within two or three years with a decent gaming rig around 2000$.
I thought you might be interested in that:

http://raytracey.blogspot.co.nz/

[youtube-hd]http://www.youtube.com/watch?v=6_DrgiwLABk[/youtube-hd]

[Mystify]

My reaction to that is "wow, that is really fuzzy"

[colocolo]

the fuzzyness derives from a stochastic algorithm called Monte carlo simulation which produces random rays.
the footage which has been showed on the web is actually not
what brigade engine can really do.
it is all kept quiet secret.

[zacherynuk]

I like that a lot. Also makes we look forward to GOT season 3

I have been waiting for hardware to allow us to move away from what I see as cheating raster tech since I bought a software 387 co-processor driver for my 386DX25 back in .. oh.. gosh '89?

If we could ray trace / path cast at decent levels it would essentially allow level creators to become true limitless architects.

I have been keeping one eye on Caustic for several years.... but we just aint there yet.

[greenknight]

the fuzzyness derives from a stochastic algorithm called Monte carlo simulation which produces random rays.
the footage which has been showed on the web is actually not
what brigade engine can really do.
it is all kept quiet secret.

Does he imply this knowledge from a blog post or youtube comment or something?

[yuriythebest]

the fuzzyness derives from a stochastic algorithm called Monte carlo simulation which produces random rays.
the footage which has been showed on the web is actually not
what brigade engine can really do.
it is all kept quiet secret.

Does he imply this knowledge from a blog post or youtube comment or something?

haha yeah. while it may sound quite fancy, it's real easy to explain. there is a way of randomly generating rays, and since there are too few rays (the sampling is too low) you get the fuzzy image. the more you increase it, the better and less grainy the image will look, however it will also eat up more frames-per-second

[Pyry]

If we could ray trace / path cast at decent levels it would essentially allow level creators to become true limitless architects.

Path tracing techniques have a number of technical advantages and disadvantages, but their adoption wouldn't change the current level design workflow very much.

[colocolo]

the fuzzyness derives from a stochastic algorithm called Monte carlo simulation which produces random rays.
the footage which has been showed on the web is actually not
what brigade engine can really do.
it is all kept quiet secret.

Does he imply this knowledge from a blog post or youtube comment or something?

The comments below imply that it could be done.

[youtube]http://www.youtube.com/watch?v=TZFM4p3l2is[/youtube]

I hope thats all true. it sounds very authentic.

[cert]

here's another brigade 2 engine demo: http://www.youtube.com/watch?v=OyY9pQEJkSk

[colocolo]

I like that a lot. Also makes we look forward to GOT season 3

I have been waiting for hardware to allow us to move away from what I see as cheating raster tech since I bought a software 387 co-processor driver for my 386DX25 back in .. oh.. gosh '89?

If we could ray trace / path cast at decent levels it would essentially allow level creators to become true limitless architects.

I have been keeping one eye on Caustic for several years.... but we just aint there yet.

http://www.blogger.com/comment.g?blogID ... 2905346881
The last comment tells if the new Caustic R2500 would accelerate
Brigade Engine.

[Diorama]

I have seen some increasingly amazing realtime ray tracing/path rendering stuff recently, take a look at some Octane Render stuff, incredible.

7 or 8 years from now....real time photorealistic graphics....on my Oculus 8K edition.......

[colocolo]

yes, octane render is great.
8K would be nice. there is a blog entry about a 17 million triangle human head rendered at 8192x4096 in 8 seconds.
http://raytracey.blogspot.co.nz/2012/12 ... ender.html

[zalo]

The primary benefits of using a raytracing approach for the Oculus rift are:
1. Prewarping will be incredibly simple and incredibly accurate (and won't add any latency).
2. Rendering an SBS 3D image won't take any more processing power than a 2D image of equivalent resolution.

I don't know what sort of tricks path tracing employs, but I imagine that these two benefits should carry over.

[Diorama]

yes, octane render is great.
8K would be nice. there is a blog entry about a 17 million triangle human head rendered at 8192x4096 in 8 seconds.
http://raytracey.blogspot.co.nz/2012/12 ... ender.html

On a GTX 680

I remember overnight render times on my old macintosh in the late 90s, for a single 800x600 raytrace render of a crummy-looking spaceship

GPU rendering like this has me very excited.

[colocolo]

The primary benefits of using a raytracing approach for the Oculus rift are:
1. Prewarping will be incredibly simple and incredibly accurate (and won't add any latency).
2. Rendering an SBS 3D image won't take any more processing power than a 2D image of equivalent resolution.

I don't know what sort of tricks path tracing employs, but I imagine that these two benefits should carry over.

well that sounds promising.

[Mystify]

"Hey, this thing looks amazing *posts video*"
"That video looks awful"
"Well, yeah, it doesn't actually look like that"

If you are going to show off how awesome something looks, then show off something that looks awesome. I'm not going to get excited about something for looking awesome if you can't show me that it does.

[jf031]

"Hey, this thing looks amazing *posts video*"
"That video looks awful"
"Well, yeah, it doesn't actually look like that"

If you are going to show off how awesome something looks, then show off something that looks awesome. I'm not going to get excited about something for looking awesome if you can't show me that it does.

Are you sure you're a developer?

[Mystify]

"Hey, this thing looks amazing *posts video*"
"That video looks awful"
"Well, yeah, it doesn't actually look like that"

If you are going to show off how awesome something looks, then show off something that looks awesome. I'm not going to get excited about something for looking awesome if you can't show me that it does.

Are you sure you're a developer?

What, asking for a video with good graphics as proof that you have good graphics is unreasonable now?

[Pyry]

I actually rather like the grainy look of sampling based global illumination techniques.

However, until a standardized API like DirectX or OpenGL is introduced that works across hardware (i.e., on more than just latest-generation Nvidia cards), these types of real time path tracing techniques aren't likely to receive widespread use in games (specialized software like CAD where workstations are bought to run specific software is a different matter). There might be a few exclusive titles optimized for particular consoles, but most of the big AAA titles have to release on all the major platforms to have any hope of recouping their development costs.

[zalo]

It could be an artistic effect, like noise or film grain...
Or even a gameplay mechanic!

Say you play as a soul or something, and you can't move too far away from special soul stones, otherwise your signal goes down, you have fewer raytraced samples, and your connection to reality diminishes. When you get too far away, the screen fades to black and you lose, while when you are near the stone, there are plenty of samples and the world is vibrant.

Lends itself to fun stories as well, but would it be too disorienting for VR?

[BOLL]

Blaghaghaalalalala! Nice videos, but one in that channel (clicked play all, chewing through it all...) had me go mind-bonkers. Embedding. Sure it's quite slow now, but what if the fillrate (if that's the right word) can be upped so much that it could be playable... zomgz...
[youtube-hd]http://www.youtube.com/watch?v=EJXWLo__PUc[/youtube-hd]

Edit: Ah, a bit quick, says Octane, not the Brigade engine then? Might have been OT... :p (haven't read up on the link provided in the description yet)

[colocolo]

Octane Render and Brigade Engine are slightly different but derive both from the same kernel.
From what ive read Octane Render doenst compromise on quality for real times sake, while in opposite Brigade a little does.

[MaterialDefender]

It might take a lot more time to make path tracing algorithms *really* work in realtime than one might think when seeing these videos. I'm working with these things as preview renderers in 3D applications on a regular basis. And while GPU based solutions are a great step speed wise compared to CPU implementations, there's still a long road ahead before this will work with good quality in realtime.

There are smarter sampling methods than brute force Monte Carlo, and that most likely is what makes things like Brigade possible in the first place, but a fundamental issue remains: the time it takes to eliminate noise increases exponentially the better the image gets, that's how these algorithms work. You can produce a recognizable but noisy image in a very short time, like shown in the demos, but an image that has a low level of noise and can be called 'final' does take much more time. No way around that.

[Randomoneh]

I've been following what Jules Urbach (OTOY) does for at least four years now. It seemed really promising in the beginning.

Does anyone know if we have computational power to fully simulate (every photon) inside of a small box (few centimeters^2) with a small light source in it? Of course, we have to guess the behavior of materials involved, but - do we have the power to simulate all those photons or not?

[Diorama]

John Carmack's opinion on real time raytracing from an interview about a year and a half ago:

http://www.youtube.com/watch?feature=pl ... nA#t=1212s (not embedded as it's 20 minutes into a half-hour interview)

Seemed cautiously optimistic.

[MSat]

I've been following what Jules Urbach (OTOY) does for at least four years now. It seemed really promising in the beginning.

Does anyone know if we have computational power to fully simulate (every photon) inside of a small box (few centimeters^2) with a small light source in it? Of course, we have to guess the behavior of materials involved, but - do we have the power to simulate all those photons or not?

I'm assuming that you're talking about in real time, correct? Within the constraints of a given model, anything can be simulated. The amount of time and memory is the limiting factor.

One day, with sufficient capacity and speed of memory, I think the "light field" of a 3D volume could be pre-calculated and an image generated in real time anywhere within that space using just a sorting algorithm. Dynamic elements would pose a challenge, but since the GPU would no longer have to render the scene (in the currently common sense), its resources could be used for calculating just the dynamic elements instead. In the near future, this might actually be well suited to servers streaming content to clients, as it would at least be a bit more efficient use of the huge memory requirements of such a system, as the number of connected clients would be largely irrelevant in terms of memory usage (they're all in the same light field), they would only add a burden when it comes to sorting, and dynamic element processing.

[Randomoneh]

Does anyone know if we have computational power to fully simulate (every photon) inside of a small box (few centimeters^2) with a small light source in it? Of course, we have to guess the behavior of materials involved, but - do we have the power to simulate all those photons or not?

I'm assuming that you're talking about in real time, correct? Within the constraints of a given model, anything can be simulated.

Let's start with what's easier - non real-time render of a scene. Has anyone in the world been able to simulate all the photon interactions in any given space? I think the amount of photon "bounces" and all the directions they bounce towards is mind-boggling. However, I want to see someone simulate it it on a small scale.

[MSat]

Does anyone know if we have computational power to fully simulate (every photon) inside of a small box (few centimeters^2) with a small light source in it? Of course, we have to guess the behavior of materials involved, but - do we have the power to simulate all those photons or not?

I'm assuming that you're talking about in real time, correct? Within the constraints of a given model, anything can be simulated.

Let's start with what's easier - non real-time render of a scene. Has anyone in the world been able to simulate all the photon interactions in any given space? I think the amount of photon "bounces" and all the directions they bounce towards is mind-boggling. However, I want to see someone simulate it it on a small scale.

I'm guessing you put the word 'bounces' in quotation marks because you know that they don't actually bounce -- they're constantly being absorbed and re-emitted, which is something that has to be considered if you're trying to model it as accurately as possible. Modelling the interaction between the photon and an atom is probably simpler than modelling an object on a macro scale (the faces of the box) and generating an image. Another tricky aspect is when do you take the "snapshot", how long of an exposure do you allow? It's the mass of photons integrated over a period of time that eventually "builds" an image. Lets say you had a camera with an ideal imaging sensor where a pixel can respond to a single photon, and all pixels are sampled simultaneously. If that imaging sensor could also have an exposure duration equal to the smallest unit of time (http://en.wikipedia.org/wiki/Planck_time), then any image captured would be very sparse in detail. Maybe just a few dimly lit pixels here and there.

I'm sure mass photon/atom interactions have been computed in a lab somewhere for the sake of physics, but probably not computer graphics. It's incredibly impractical, and not nearly as straight forward as ray tracing makes it seem. Still, while it may be out of the realm of practicality, it's not out of the realm of possibility

[zalo]

As far as fun real time demos simulating photon bounces (in a box) goes, this real time java applet has always been my favorite: http://www.cc.gatech.edu/~phlosoft/photon/

It's good looking, fast, and the code for that whole program is only a couple hundred lines long (and brain-dead simple to understand because it's all commented so well).

[Pyry]

Well, keep in mind real light exhibits both wave and particle behavior, and simulating the wave aspects gets very computationally intensive very quickly. However, if you're willing to ignore all the wave stuff, and just treat light as particles, then the energy of a 500nm wavelength photon (greenish light) is approximately 4*10^-19 J (equation here: http://www.wolframalpha.com/input/?i=en ... wavelength). If you wanted to simulate all the photons emitted by, say a 60W green light bulb, then in one frame (1/60th of a second) you would have to simulate 60J/s * (1/60)s = 1J of photons, which would be 1J/(4e-19 J/photon) = 2.5*10^18 photons.

Assuming that you could simulate each photon in just one floating point operation (not even remotely close to the truth, but we're being generous), then you would need on the order of 50,000,000 GTX 680s (each at approximately 3000 gflops according to http://en.wikipedia.org/wiki/Comparison ... 600_Series) to do it in real time.

[colocolo]

here is a quick update on how is Brigade engine 2 is progressing.
Brigade team has worked now for 2 months on a big demo which they are going to show at GTC in San Jose in a couple of weeks.
After the show there will be noisefree 1080p videos on youtube with 15- 20 frames how i figured out.

quote from
http://www.blogger.com/comment.g?blogID ... 2905346881

colocolo said...
Hey Anonymous, there haven't been much updates lately because we're working very hard on a tech demo with Brigade that we'll be showing in a couple of weeks, and I don't want to spoil the surprise. It will blow your mind, I will post lots of screenshots and videos when the show is over.

Will we see a 1080p demo or at least 1080p screenshots? you are allowed to tell this,or not?

March 7, 2013 at 5:19 AM

Sam Lapere said...
colocolo: Yes there will be 1080p screens and videos. We did some tests this and last week and Brigade runs incredibly well in 1080p. It's as if you're watching a movie that you can play, I was blown off my socks how good it looks.

[Diorama]

Thanks for the update! I am really excited about the Brigade 2 engine, any news is very welcome! The videos I have seen blew me away. I may have said it before but I remember rendering a spaceship in StrataVision3D and it took all night. For one 800x600 image.

[Hermit]

You guys must be joking, this kind of magic is not possible.

[geekmaster]

You guys must be joking, this kind of magic is not possible.

Believe in the impossible, by Lewis Carroll

"I can't believe that!" said Alice.

"Can't you?" the queen said in a pitying tone. "Try again, draw a long breath, and shut your eyes."

Alice laughed. "There's no use trying," she said. "One can't believe impossible things."

"I dare say you haven't had much practice," said the queen. "When I was your age, I always did it for half an hour a day. Why, sometimes I've believed as many as six impossible things before breakfast."

Clarke's three laws, by Arthur C. Clarke

When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

Any sufficiently advanced technology is indistinguishable from magic.

[mattyeatsmatts]

I hope this is better than that unlimited detail engine that Euclideon tried to make us believe in

[cybereality]

That sci-fi video looks amazing. Clearly nowhere close to real-time, but still very impressive.

[Scog]

http://raytracey.blogspot.co.nz/

The latest video including animated characters is very impressive. Can't wait to see the new demo coming soon.

[Paintbrush]

Impressive!
I think we are still a few years away from this being really useful in a game. But we are slowly getting there.
If this is the future, then it'll be amazing one.

[Hermit]

A few years is not nearly enough. This method takes miliseconds per pixel on todays high end consumer hardware. for vr we want a maximum of 8ms per frame on 1080p!

[Linkage1992]

A few years is not nearly enough. This method takes miliseconds per pixel on todays high end consumer hardware. for vr we want a maximum of 8ms per frame on 1080p!

One of the guys working on this said that within a year commercially available AMD chips would be able to render most of these scenes in real time with no grain, and that was a few months ago. Progress is happening faster than most people realize.