A day with an Oculus Rift

[ThomasC]

Completely agree about the potential inaccuracy of the LOD view (as well as the associated issue of collapsing the depth), but on the flip side, we're talking about a failsafe that's only there as a placeholder until the primary render can catch up (hopefully less than 1/30 of a second later), and one which is likely being blurred due to fast movement.

I'd argue that by the same token (though admittedly maybe to a lesser extent), texture streaming and adaptive LODs can look "wrong" too, but they can also be better than the alternatives.

I think we're likely to run into issues rendering a "fast mode", given the complexity and variety of engine design and their inherent threading and internal rendering latencies. I.e. a fast turn that pivots you into a wall still needs to reconcile the viewpoints against the collision in order to prevent you crashing through the world, so now you have to wait on the app and the followup render on top of hardware latency involved.

On top of that, we're now putting the onus on the developer to manage the issues being imposed on them by a new target platform, whereas abstracting as many issues as possible to the device could potentially get more groups on board. It's fantastic that John is all-in with customizing their work, but I'm sure we all agree that in the long run most of these details need to be abstracted away from the developer.

On almost every game I've worked on, we've gone in with the ambition to run at 60 fps (consoles), but at the end of the day, when you're looking longingly at those extra 16 ms, it's hard to give up all the bonus features you could have had. Trying to convince developers to write their games to run in one or two binary orders of magnitude less time in order to support an interesting but initially small market is going to be a hard sell.

[PalmerTech]

5. It's unreasonable to expect software developers to account for per-display variations (i.e. distortion mappings) within their software (i.e. currently the unique needs of a display are handled by the display itself and applications are expected to interface with displays through a set of standards (resolution and physical connection)).

What this ultimately leads to, is the notion that the game/software and it's requisite hardware really should be abstracted from the needs of the display.

I definitely agree. One of the goals for Oculus is to have an SDK that is easy to integrate, and once it is integrated, hardware flexible. The idea is that the hardware could report its specs when you plug it in (FOV, resolution, tracking, warping parameters, etc), and the Oculus integration would adjust the software to fit. That way, the hard part of all this would be on Oculus, not on developers. You are also right about the performance vs features issue, we have a few ideas on how to mitigate that as much as possible. One good thing is that we don't have to worry all that much about 180 snaps, people can't turn very fast with their heads compared to a mouse! It will take lots of time and expertise brainpower to get everything perfect, but that is exactly what the Rift can be used for: Getting things perfect for a consumer version.

[ThomasC]

I'm totally on-board with what you guys are doing, which I hope will ultimately validate VR as a "next big step". Most of my line of thought has come from thinking about how good AR might ultimately work. If we suppose that some day we end up with something along the lines of a synthetic aperture light field camera (maybe some kind of interferometer?) which can generate real time high fidelity depth information, then the next issue will be how to receive that depth input from the camera, merge it with CG, and re-display it fast enough to work.

In this particular case, it's pretty evident than any mis-registration of the graphics over the live view is going to completely destroy the illusion of the two belonging to the same world. Once you start pursuing that line of thought, you start realizing how critical bringing latency as near to zero as possible is going to be. I suspect that in reality we'll need to both reduce latency and either do something similar to current CG motion tracking techniques based off of scene contrast, or "cheat" by capturing the live video and fully reprocessing it (i.e. not actually blending CG into a live shot, but actually encoding the live view and re-imaging it interleaved with the CG).

The purpose of my butting into this thread was ultimately aimed farther down the road, once the various features are positioned as the responsibility of either the app or as specialized hardware sitting beside the display. In my mind, if a low powered SoC containing a reasonably powerful gpu could be coupled directly to the motion tracker, then by providing the primary rendered image from the app with a guardband around it, you could conceivably take a lot of the loop out of the equation and end up with blazing fast tracking by letting specialized hardware drift the view locally within the display (the LOD environment I posited was really just an extension of this line of thought... perhaps getting ahead of myself). You guys are currently focused on VR, not AR, but I can easily see your concerted push creating a set of standards and I felt it couldn't hurt to float a couple of longer term concerns.

I do still think it's worth considering, that most players are quite happy with a 60hz simulation, and that 120/240 hz will be critical for tracking believability. In my mind this implies decoupling the two by some mechanism similar to what I'm imagining in my head, but I suspect you guys have thought this through already.

[brantlew]

A few minutes with an Oculus Rift

Well I finally got to try the Rift. I was at QuakeCon Thursday and half of Friday. Since Oculus was such a recent addition to the show, there were a few technical and bureaucratic issues that prevented Palmer and crew from demoing the Rift on Thursday. However, Palmer graciously granted me and another MTBS3D member a private demonstration on Thursday night. Now I didn't get to actually see the Doom 3 demo, but I did see an earlier Carmack test within a small Rage environment. On Friday, the Doom 3 demo was still unavailable so they were still showing the same Rage demo to the public. So I got to see it twice, but in total only for about 5 minutes. Not near enough for a proper and valid review - but I gotta talk about it anyway.

It's hard to be very objective. The first few minutes is really all about just wanting to look around. It's hard to really concentrate on the details: are the edges visible, can I see the pixel structure, what is the resolution like - all the stuff that I wanted to look at closely just got shelved as soon as I stuck my face in the Rift, because the coolness of the whole thing is just so overwhelming. You just can't stop looking around and admiring just how "real" it all feels. At one point I found myself just grasping my hand in midair because it looked so much like there was a cartoon pipe right in front of me. The sense of depth is just amazing - so far beyond 3D (but not in a silly pop-out way). Everything just somehow has a tangible "weight" and depth to it. When you move to the edge of a ledge and look down - you feel the vertigo in your gut. I experienced simulator sickness for the first time - not just some general discomfort. But a strong and instant gutteral feeling as I was looking down a hole and swaying back and forth. Oh, and for you guys that are concerned about the resolution... With the strong antialiasing, it didn't bother me at all. Now I can certainly imagine for reading text and HUDs it would be noticeable, but with pure scenery watching - it is not a big deal. I forgot to even try to find the pixels because I was so enthralled with the experience.

If I have to nitpick, I would say my main issue was a small tracking latency that I observed. One of the major points that came across to me at QuakeCon was the important of low latency. I've heard Carmack talk about it endlessly and I sort of discounted what he was saying a bit - assuming he was just obsessing over the last 2%. But when it comes to this level of immersion I completely understand his point now. You can get away with all kinds of delays and inaccuracies with non-immersive displays. But the moment you start to feel like you are "in there", you can't ignore those things anymore. I believe Carmack in the keynote claimed the Doom 3 Rift latency was around 40 or 50 milliseconds. Now I saw an older version of that code base, so it may have been even a bit more on this demo. But it was definitely noticeable to me. I wouldn't call it a "stutter" necessarily - that's overstating it. More like a "vibration" as I panned my head. You wouldn't think about it twice if it was your frame rate on a normal screen. But on the Rift, the effect is amplified.

And that's one potential pitfall I see with the Rift. The device is so good, that it amplifies any other problems with latency and inaccuracy. It forces perfection in every other aspect of the simulation. Forget trying to play games with all the effects turned up and running at low frame rates. You'll need all the frame rate you can muster. And it forces me to seriously reconsider inaccuracies in my own projects. Currently I can tolerate all types of motion inaccuracies, stuttering, and latency problems. With the best consumer HMDs those problems just sort of look crummy, but I can deal with it. But I suspect the Rift is not so forgiving - and instead of just looking crappy it might actually make me throw up! Another subtle detail - the way that Carmack modeled the head translation as you rolled your head was sort of funny. The first time I tilted my head sideways the wall in front of me sort of stretched and sheared. The reason is because I was rotating my head around my chin (sort of lopping it to the side onto my shoulder). Well the Carmack interpretation was more like rotating around my nose. Once I consciously rotated around my nose, everything looked correct. That's the sort of thing that you would never-ever notice on a desktop screen, but is so obvious on the Rift.

Ok, well I've gone on long enough - much longer than I actually even used the device. To sum it up, I think it's just fantastic. I would be completely satisfied even if this was the consumer version. Well done Palmer. I can't wait to get one at home to start tinkering with it.

[WiredEarp]

Awesome brantlew, its really nice to get an objective opinion on the Rift.

You just can't stop looking around and admiring just how "real" it all feels.

I was like that on the first virtuality system I used, looking at the gun in my hand, and the fact that it felt almost like putting a scuba goggle on and going diving. With higher FOV, you really get a sense of 'being there' that you dont get with lower FOV HMD's.

The latency thing has ALWAYS been an issue with VR unfortunately. I wonder how much extra latency CyberReality and emersons warping drivers will introduce as well?

Re the translation issue, I think this is probably more of a concern due to the choice of tracker. I'd imagine we were using an absolute tracker like a magnetic one, then roll and rotation should work much more naturally.

[cybereality]

@brantlew: Thanks man. Sounds pretty good to me.

@WiredEarp: The driver certainly has overhead but I haven't done any benchmarks yet. The main issue right now is that I am rendering the eyes sort of like page-flipping (ie frame 1 is left, frame 2 is right) but I am not caching draw calls. So basically it means if you want the full 60FPS in 3D you need to be running at over 120FPS with vsync off. For games like HL2 and L4D this is not a problem and looks fine. But with more intensive games it would be a problem. Beyond that, when I integrated the Hillcrest tracker the framerate dropped a bunch. I think if I pull it into a separate thread that would speed things up, but right now the tracking seems intensive.

[FingerFlinger]

Does anybody remember what the Rift's latency is, from video cable to screen? I vaguely remember JC saying that he measured it, but I can't recall the number.

EDIT: He mentioned during the keynote, it's between 40 and 50ms.

[OmniAtlas]

I'll be investing with your kickstarter project -- I'm excited. Gaming aside, I would be interested in using this with flight simulation (Microsoft FSX). As you know, FSX is an ancient program and development stopped a longtime ago. It is largely the community and third party software which keeps it alive (check out http://fullterrain.com for beautiful realistic scenery, and PMDG http://www.precisionmanuals.com for a very realistic rendition of the Boeing 737-800).

Right now tracking can be done within the cockpit with hardware such as TrackIR (leds are mounted to your headset or your cap) and the software directly interprets the coordinates for use within FSX.

Would there be a way to develop a software so Oculus Rift could be universally used with software that do not have active developer support?

There are already drivers that can load into the background to perform what Oculus Rift may need to offer full immersion for all programs --

DIY Stereo 3D Driver: http://www.youtube.com/watch?v=Ovf5TLiIfZ8
Image Warping: http://www.fly.elise-ng.net/index.php/i ... splaylite2 (I believe this directly interfaces with d3dx9.dll)

[brantlew]

@OmniAtlas: There are a couple of developers on this site that are working on drivers to convert preexisting games to the Rift format. One of them is open source. I would check there.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15086

http://www.mtbs3d.com/phpBB/viewtopic.php?f=26&t=14970

[BillRoeske]

A few minutes with Doom 3 on the Rift

As a lot of you know, Palmer and company were able to demo the E3 version of Doom 3 BFG for the public with the Rift on Saturday at QuakeCon. The short version is that it is awesome and demoed quite well. The hall opened at 9:00am, and by 11:30am the line was already an hour long. Occasionally people who were coming off the demo station would shout back over to the rest of the line, "it's worth it!" I had to wonder how many of them realized that some of us had been standing in that line for twenty years.

First up, some thoughts on the hardware.

They were actually using JohnCarmack's modification of the Rift (as seen in his E3 interviews with the blinders, black tape on the lens edges, and single elastic ski goggles band to hold it to your head). I was actually pretty happy to see that since it closely resembles the developer kit design. It felt reassuringly secure and lightweight. A few of my friends complained that it sat just a little too close (they felt their eyelashes brushing the plastic lenses), but I didn't have that problem. Lack of ventilation could be an issue for long-term use, but the blackout effect is really, really effective. Time to start researching DIY ways to allow air in, but not light?

I'm near-sighted and didn't have any trouble getting a focused image, but for my other friend (that requires glasses for pretty much everything), it was basically unusable. The glasses question was on everyone's mind at QuakeCon and I can understand Palmer's hesitation to create much more space between the eye and the lens; to do so would diminish the FOV pretty quickly. Larger lenses could fix that, but then you're in the game of trying to source new parts. Palmer has pitched a few ideas in interviews, so we'll see what they ultimately settle on there.

The display panel is probably the weakest part of the package. Resolution is on everyone's mind, so I'll get that out of the way first: it honestly wasn't a big deal. Yes, the grid and sub-pixels are there when you look for them. Yes, I will happily trade up when a higher-resolution panel is available. As it was, though, the grid was gone whenever I became engaged. My wish is for a panel with better contrast and a faster response time. Those two issues combined flattened out the feeling of depth somewhat, and more than the resolution, reminded me that I was in a simulation. Happily, the panel is also the part with time (and the massive arms race of the mobile industry) on its side. The panel that's there should be perfectly adequate for a developer kit, and even a heck of a good time in Doom 3. In the mean time, I would just author content on a traditional monitor and trust that the display in consumer HMDs will be up to spec.

Playing Doom 3 itself was great fun. Even though my friends and I knew that there wasn't any positional tracking, we still couldn't help dodge to the side in our chair a little bit as a fireball arced toward us. The segment on demo was from a very action-heavy part of the game when you have a drone helping you take down the constant stream of imps. It's a smart choice since it's relatively hard to die, and the player has some time to get oriented and be a bit of a tourist. Having a game pad input mapped to handling large turning worked well for seated play, as did decoupling the pitch of the weapon from the view.

I'm used to seeing VR demos with terrible graphics, so playing a polished game was actually a real treat. The Rift and Doom 3 do enough things noticeably better than anything else at the consumer level that I felt like I was experiencing a lot of "firsts" all over again.

[BillRoeske]

Some other random impressions:

• The increase in vertical FOV feels almost more important than the increased horizontal in terms of making me feel grounded in a world. It practically filled my eye's entire FOV for looking straight ahead.
• There was a slight amount of perceivable latency in Doom 3. Overall, it's really really good. But I just want to set some expectations.
• Smoke planes, light-bending teleportation, and other "2D" effects stood out, of course.
• The 3D effect is really pronounced. Not in an artificial way, mind you. Just in a "I need to consciously not try reach out to grab that steam valve" way. I try not to give in to hyperbole, but I actually did have that moment with the Rage demo.
• Good VR is addictive.

[Zaptruder]

Thanks for the feedback guys.

I'm prepared to get contacts to give this thing a good go!

[brantlew]

Thanks Bill. Did you notice a difference in tracking responsiveness between the Rage demo and Doom demo? Carmack talked about doing some sensor forwarding in the rendering pipeline to reduce the delay. I don't know whether that was present in the Rage demo, and I was curious whether you noticed anything.

Edit:

[*]The 3D effect is really pronounced. Not in an artificial way, mind you. Just in a "I need to consciously not try reach out to grab that steam valve" way. I try not to give in to hyperbole, but I actually did have that moment with the Rage demo.

I had the same impression, and I think I tried to grab that same pipe.

[BillRoeske]

Did you notice a difference in tracking responsiveness between the Rage demo and Doom demo? Carmack talked about doing some sensor forwarding in the rendering pipeline to reduce the delay. I don't know whether that was present in the Rage demo, and I was curious whether you noticed anything.

I absolutely noticed a difference between the two. Doom felt better, but I'm pretty sure it also was running on a better PC at a higher, more consistent frame rate than the Rage snippet.

[cybereality]

@BillRoeske: Thanks for the reassuring words, though I was completely sold a long time ago...

[ktommy]

I'm cross-eyed, about 20 degrees. Do you think there would be enough adjustability to get the two images into my field? For an example, most binoculars won't go close enough together for me. Am I right in understanding that at least some of the adjustment in the horizontal is done in software? Is the hardware in the present user-assembled model potentially adjustable by a someone who likes to play with optics? --- Thanks for this great, nuts and bolts discussion.

[2EyeGuy]

I think the kinds of lenses they are using allow for a range of eye positions.
Software-wise, you probably want the separation/convergence adjusted so that the centre of each eye's display is closer to your nose (off-axis rendering). I'm not sure if you'd want the cameras to be angled in or not.

[ktommy]

Thanks for the reply. I hope a convention comes to N Calif in the next few months. I want to try one.

[mickman]

What an amazing read...

FInally I climb out from under my rock.. nearly 20 years have past since I donned a VFX1.

This time they have my full attention ... I am on my knees praying this will usher in the greatly awaited age of VR.

I think I hit the BUY NOW button , after only three minutes of reading over the Occulus Rift site.

This will be the greatest Christmas ever !

Thanks Palmer & John for finally lifting the game