If the kickstarter hits a certain point plus having a major company like ID showing interest I'm sure companies like LG or Samsung should look your way.

I agree a software bundle with the HMD that can show the strength of gaming using independent head tracking would be a great selling point. Also, 1920x1080 panel would be a better match as I could use most off the shelf equipment for wireless HDMI gaming.

With Samsung now starting to push 1080p panel for handphone and tablet, maybe can checkout if they sell any demo kits.

Wow. This is almost perfect.

I just hope they are still avilable when I have the cash.

(To give an idea of the recognition, even my girlfriend knows quake)

If JohnCarmack was to make a demo that runs on this unit, OH MY...Match made in heaven.
I think a one level JohnCarmack demo run on this unit would have you playing that one level, over and over and over..ect.
People would buy the RIFT, just to play ID games, if they were the only company smart enough to write code for it.
The unit already has a sort of. bad AZs doom look to it...LOL

I know I would.

Hi, I have just one question, instead of asking "please samsung" why don't you just get a galaxy S3 and stick that into the glasses? Super Amoled 4.8” 1280x720 screen, powerful 1.4ghz quad core cpu with 3d acceleration, 1 gig ram, it should be enough to get a quite decent game runing on it controlled with a bluetooth joystick like the PS3 one (that has accel sensors too), and should have plenty of power to handle wireless receiving and decompression of live video from a PC if you need so... It's also self contained, has it's own battery, audio out, gyro and accelerometers, and even a very nice high fps and high res camera on the back that you can use for head tracking. Maybe you can even ditch android if it eats too much resources and run directly on linux...
Yes, it cost a bit too much, but as soon as the price drops a bit it should be a very good option.
What do you think about it?
My best regards to you guys, you're making my dreams come true, and a special thank to John for just being awesome, I'm a huge fan of your work, you made history!!
Can't wait to get my pair of VR glasses.

I would imagine latency to be an issue with that. Weight is also a consideration.

Some discussion of using phone screens for HMD.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=26&t=14365&hilit=phone

That discussion gets to my same conclusion, now that the Galaxy 3 is out! It looks like the S3 has the 802.11n wifi, not the 5ghz one, so it may have a big lag indeed, but they claim that they can transfer a 1gb file in 3 minutes via the NFC connection, so that may help. Doing all the processing on the phone is not a bad thing anyway, with such power available.

Another possibility

http://arstechnica.com/gadgets/2012/06/ ... ckstarter/

Maybe not this specific device, but I'm sure others with more graphics power will be created.

The screen is a bit too small, they use a pentile arrangement for the sub-pixels that looks awful under magnification, and even the fastest streaming out there has about 50ms of lag, and that is with pretty heavy compression that makes the game look terrible.

Not a very good option right now, but in a few years, I think most HMDs will be rendering their games with onboard ARM based SOCs anyways.

I'm certainly hoping to try hook some of our Android devices with HDMI-out to the Rift when I get my hands on one. Given the somewhat flaky support of HDMI-out on those devices, I wouldn't be surprised if there's some increased latency above the already floaty timing of eglSwapBuffers() Android. I'll be interested to see what shakes loose, at the very least.

I'm going to try to snag a Hillcrest tracker/Doom 3 bundle, so I'll probably see what can be done about working with libfreespace on Android, too. That will be completely new territory for me, so hey, adventure!

Ideally I'd like to work up something that's useful for other people, too. Depending on the the form that John's example code takes (and if he's given permission to release it), the most productive thing to do might be to ultimately release an Android port of that.

I agree with both quotes, however I'd rather not have HMD's rendering with their own SOC's but rather using our existing devices, iOS, Android (WP8 anyone?) and work with those guys to reduce latency, lag, etc... in their pipelines. They all recognize the value of games and there are many high profile engines out there that it shouldn't be a problem to get them to make some changes to accommodate this lower latency which it looks like John has made into a personal quest anyways Here's to this happening sooner rather than later!

</delurk>
Apologies in advance for poking my nose in... I'm a long time forum lurker.

I'd like to throw some potentially important considerations out there in regards to future Rift development; in my reading the various threads I've not seen this concept brought up (again, apologies if it has).

If we start with a few basic assumptions this might help frame some concerns.

1. John C. is spot on that update latency and frequency are critical to immersion in a VR device.
2. Update frequency/latency is possibly even more important in an AR (vs VR) system where any registration problems for the CG overlayed with the real time video will immediately be immersion destroying (I only bring this up because with any luck, development of the Rift could become a driving force towards VR device interface standards).
2. A VR/AR system should be able to handle sudden changes in behaviour (i.e. the player snap-turns 180 degrees), this introduces the need for a system similar to texture streaming where a low fidelity representation is always available to cover for us.
3. Simulation/Rendering update frequency is not necessarily the same thing as display update frequency. Where 60hz is a pretty reasonable simulation update, I'm sure we'd all love to see 240hz motion tracking/camera frustum updates.
4. It's potentially unreasonable to expect game developers to put out games that complete their simulation/rendering in a sub 4ms time frame (or 8ms for a 120 hz update), as you're leaving a lot of other potentially cool features on the table in order to service the display device fast enough (physics, better AI, etc).
5. It's unreasonable to expect software developers to account for per-display variations (i.e. distortion mappings) within their software (i.e. currently the unique needs of a display are handled by the display itself and applications are expected to interface with displays through a set of standards (resolution and physical connection)).

What this ultimately leads to, is the notion that the game/software and it's requisite hardware really should be abstracted from the needs of the display.

In the long run, we might be better served by taking the approach that any given app should simply provide the same information to a VR display that it currently does, at an expected update rate of 30 (boo) to 60 (yay!) hertz (with one potential caveat regarding a potential future standard) with a standardized resolution.

This would imply a two stage system whereby the application runs as usual on its own hardware, and the submitted frame is then passed to a second, specialized hardware system whose job is to update the display orientation and frame presentation taking into account a visual "guardband" around the expected view direction.

To take this one step farther, such a system would ideally be capable of showing a low fidelity 360 degree placeholder of the current scene (possibly as a cube map) which would be cross faded with the high fidelity main view dependant on the player's movement.
By "feathering" the edge of the primary display into the low fidelity surrounding environment, we create a soft transition that removes any hard edge artifacts between the primary and secondary draw layers.

As a bonus, by running this daughter system, we gain a place to handle any VR/device specific needs, like applying hardware specific display distortion maps and computing motion blur all in one pass.

In this design, the app provides the current simulation frame to the daughter system in its own time frame, and the VR's secondary hardware simply flips the simulation image whenever it's ready. A daughter system of the required horsepower/size likely exists in the form of contemporary SOCs used in modern smart phones, and as such could be made small enough and low power enough to reside in the hardware itself.

In regards to the LOD environment.

This is the sticky part that seems to necessitate some custom development by the software developer (hopefully someone has a better idea). If a standard was adopted by which a portion of the HDMI signal could contain an encoded cube map which the daughter system could decode, then software developers could implement a low frequency update to this secondary VR environment (i.e. at 1/6 the current rendering speed) by locking a view point and rendering a single cardinal view per frame, then encoding the resulting cube faces into the frame (possibly stacked vertically along one side of the render frame and decoded by the daughter hardware?).



I don't want to overthink this, nor do I feel this is necessarily an answer to the concerns I've listed, but I was hoping it might be useful to bring these thoughts up in the interest of promoting a discussion that takes many of the Occulus Rift concepts further along the directions they're already headed. I'd also like to point out that the current approach is immediately supportable, and that I submit these thoughts more as future consideration than something that should be dealt with right now. Now is the time to hack =).


I've been following threads on VR devices here on MTBS for a long time, and my fascination with VR goes back 20 years (I owned a Virtuality system at one point... (don't ask) )...
I'd like to chip in my personal appreciation at all of you who are helping to move the ball forward in regards to pushing the technology and standards...

All the best,

TC.

</relurk>

The LQ 360deg render technique won't work with a freely moving HMD. Turning your head does not just rotate your eyeballs in place individually, but moves both of them around a separate common centre of rotation, which itself is likely to be translating in multiple axes. You would have to pre-render a scene for multiple possible eye-pair positions and interpolate between them to get any benefit over just updating the view direction and sending the engine a 'render fast' flag for very rapid movements.

Completely agree about the potential inaccuracy of the LOD view (as well as the associated issue of collapsing the depth), but on the flip side, we're talking about a failsafe that's only there as a placeholder until the primary render can catch up (hopefully less than 1/30 of a second later), and one which is likely being blurred due to fast movement.

I'd argue that by the same token (though admittedly maybe to a lesser extent), texture streaming and adaptive LODs can look "wrong" too, but they can also be better than the alternatives.

I think we're likely to run into issues rendering a "fast mode", given the complexity and variety of engine design and their inherent threading and internal rendering latencies. I.e. a fast turn that pivots you into a wall still needs to reconcile the viewpoints against the collision in order to prevent you crashing through the world, so now you have to wait on the app and the followup render on top of hardware latency involved.

On top of that, we're now putting the onus on the developer to manage the issues being imposed on them by a new target platform, whereas abstracting as many issues as possible to the device could potentially get more groups on board. It's fantastic that John is all-in with customizing their work, but I'm sure we all agree that in the long run most of these details need to be abstracted away from the developer.

On almost every game I've worked on, we've gone in with the ambition to run at 60 fps (consoles), but at the end of the day, when you're looking longingly at those extra 16 ms, it's hard to give up all the bonus features you could have had. Trying to convince developers to write their games to run in one or two binary orders of magnitude less time in order to support an interesting but initially small market is going to be a hard sell.

I definitely agree. One of the goals for Oculus is to have an SDK that is easy to integrate, and once it is integrated, hardware flexible. The idea is that the hardware could report its specs when you plug it in (FOV, resolution, tracking, warping parameters, etc), and the Oculus integration would adjust the software to fit. That way, the hard part of all this would be on Oculus, not on developers. You are also right about the performance vs features issue, we have a few ideas on how to mitigate that as much as possible. One good thing is that we don't have to worry all that much about 180 snaps, people can't turn very fast with their heads compared to a mouse! It will take lots of time and expertise brainpower to get everything perfect, but that is exactly what the Rift can be used for: Getting things perfect for a consumer version.

I'm totally on-board with what you guys are doing, which I hope will ultimately validate VR as a "next big step". Most of my line of thought has come from thinking about how good AR might ultimately work. If we suppose that some day we end up with something along the lines of a synthetic aperture light field camera (maybe some kind of interferometer?) which can generate real time high fidelity depth information, then the next issue will be how to receive that depth input from the camera, merge it with CG, and re-display it fast enough to work.

In this particular case, it's pretty evident than any mis-registration of the graphics over the live view is going to completely destroy the illusion of the two belonging to the same world. Once you start pursuing that line of thought, you start realizing how critical bringing latency as near to zero as possible is going to be. I suspect that in reality we'll need to both reduce latency and either do something similar to current CG motion tracking techniques based off of scene contrast, or "cheat" by capturing the live video and fully reprocessing it (i.e. not actually blending CG into a live shot, but actually encoding the live view and re-imaging it interleaved with the CG).

The purpose of my butting into this thread was ultimately aimed farther down the road, once the various features are positioned as the responsibility of either the app or as specialized hardware sitting beside the display. In my mind, if a low powered SoC containing a reasonably powerful gpu could be coupled directly to the motion tracker, then by providing the primary rendered image from the app with a guardband around it, you could conceivably take a lot of the loop out of the equation and end up with blazing fast tracking by letting specialized hardware drift the view locally within the display (the LOD environment I posited was really just an extension of this line of thought... perhaps getting ahead of myself). You guys are currently focused on VR, not AR, but I can easily see your concerted push creating a set of standards and I felt it couldn't hurt to float a couple of longer term concerns.

I do still think it's worth considering, that most players are quite happy with a 60hz simulation, and that 120/240 hz will be critical for tracking believability. In my mind this implies decoupling the two by some mechanism similar to what I'm imagining in my head, but I suspect you guys have thought this through already.

A few minutes with an Oculus Rift

Well I finally got to try the Rift. I was at QuakeCon Thursday and half of Friday. Since Oculus was such a recent addition to the show, there were a few technical and bureaucratic issues that prevented Palmer and crew from demoing the Rift on Thursday. However, Palmer graciously granted me and another MTBS3D member a private demonstration on Thursday night. Now I didn't get to actually see the Doom 3 demo, but I did see an earlier Carmack test within a small Rage environment. On Friday, the Doom 3 demo was still unavailable so they were still showing the same Rage demo to the public. So I got to see it twice, but in total only for about 5 minutes. Not near enough for a proper and valid review - but I gotta talk about it anyway.

It's hard to be very objective. The first few minutes is really all about just wanting to look around. It's hard to really concentrate on the details: are the edges visible, can I see the pixel structure, what is the resolution like - all the stuff that I wanted to look at closely just got shelved as soon as I stuck my face in the Rift, because the coolness of the whole thing is just so overwhelming. You just can't stop looking around and admiring just how "real" it all feels. At one point I found myself just grasping my hand in midair because it looked so much like there was a cartoon pipe right in front of me. The sense of depth is just amazing - so far beyond 3D (but not in a silly pop-out way). Everything just somehow has a tangible "weight" and depth to it. When you move to the edge of a ledge and look down - you feel the vertigo in your gut. I experienced simulator sickness for the first time - not just some general discomfort. But a strong and instant gutteral feeling as I was looking down a hole and swaying back and forth. Oh, and for you guys that are concerned about the resolution... With the strong antialiasing, it didn't bother me at all. Now I can certainly imagine for reading text and HUDs it would be noticeable, but with pure scenery watching - it is not a big deal. I forgot to even try to find the pixels because I was so enthralled with the experience.

If I have to nitpick, I would say my main issue was a small tracking latency that I observed. One of the major points that came across to me at QuakeCon was the important of low latency. I've heard Carmack talk about it endlessly and I sort of discounted what he was saying a bit - assuming he was just obsessing over the last 2%. But when it comes to this level of immersion I completely understand his point now. You can get away with all kinds of delays and inaccuracies with non-immersive displays. But the moment you start to feel like you are "in there", you can't ignore those things anymore. I believe Carmack in the keynote claimed the Doom 3 Rift latency was around 40 or 50 milliseconds. Now I saw an older version of that code base, so it may have been even a bit more on this demo. But it was definitely noticeable to me. I wouldn't call it a "stutter" necessarily - that's overstating it. More like a "vibration" as I panned my head. You wouldn't think about it twice if it was your frame rate on a normal screen. But on the Rift, the effect is amplified.

And that's one potential pitfall I see with the Rift. The device is so good, that it amplifies any other problems with latency and inaccuracy. It forces perfection in every other aspect of the simulation. Forget trying to play games with all the effects turned up and running at low frame rates. You'll need all the frame rate you can muster. And it forces me to seriously reconsider inaccuracies in my own projects. Currently I can tolerate all types of motion inaccuracies, stuttering, and latency problems. With the best consumer HMDs those problems just sort of look crummy, but I can deal with it. But I suspect the Rift is not so forgiving - and instead of just looking crappy it might actually make me throw up! Another subtle detail - the way that Carmack modeled the head translation as you rolled your head was sort of funny. The first time I tilted my head sideways the wall in front of me sort of stretched and sheared. The reason is because I was rotating my head around my chin (sort of lopping it to the side onto my shoulder). Well the Carmack interpretation was more like rotating around my nose. Once I consciously rotated around my nose, everything looked correct. That's the sort of thing that you would never-ever notice on a desktop screen, but is so obvious on the Rift.

Ok, well I've gone on long enough - much longer than I actually even used the device. To sum it up, I think it's just fantastic. I would be completely satisfied even if this was the consumer version. Well done Palmer. I can't wait to get one at home to start tinkering with it.

Awesome brantlew, its really nice to get an objective opinion on the Rift.



I was like that on the first virtuality system I used, looking at the gun in my hand, and the fact that it felt almost like putting a scuba goggle on and going diving. With higher FOV, you really get a sense of 'being there' that you dont get with lower FOV HMD's.

The latency thing has ALWAYS been an issue with VR unfortunately. I wonder how much extra latency CyberReality and emersons warping drivers will introduce as well?

Re the translation issue, I think this is probably more of a concern due to the choice of tracker. I'd imagine we were using an absolute tracker like a magnetic one, then roll and rotation should work much more naturally.

@brantlew: Thanks man. Sounds pretty good to me.

@WiredEarp: The driver certainly has overhead but I haven't done any benchmarks yet. The main issue right now is that I am rendering the eyes sort of like page-flipping (ie frame 1 is left, frame 2 is right) but I am not caching draw calls. So basically it means if you want the full 60FPS in 3D you need to be running at over 120FPS with vsync off. For games like HL2 and L4D this is not a problem and looks fine. But with more intensive games it would be a problem. Beyond that, when I integrated the Hillcrest tracker the framerate dropped a bunch. I think if I pull it into a separate thread that would speed things up, but right now the tracking seems intensive.

Does anybody remember what the Rift's latency is, from video cable to screen? I vaguely remember JC saying that he measured it, but I can't recall the number.

EDIT: He mentioned during the keynote, it's between 40 and 50ms.

I'll be investing with your kickstarter project -- I'm excited. Gaming aside, I would be interested in using this with flight simulation (Microsoft FSX). As you know, FSX is an ancient program and development stopped a longtime ago. It is largely the community and third party software which keeps it alive (check out http://fullterrain.com for beautiful realistic scenery, and PMDG http://www.precisionmanuals.com for a very realistic rendition of the Boeing 737-800).

Right now tracking can be done within the cockpit with hardware such as TrackIR (leds are mounted to your headset or your cap) and the software directly interprets the coordinates for use within FSX.

Would there be a way to develop a software so Oculus Rift could be universally used with software that do not have active developer support?

There are already drivers that can load into the background to perform what Oculus Rift may need to offer full immersion for all programs --

DIY Stereo 3D Driver: http://www.youtube.com/watch?v=Ovf5TLiIfZ8
Image Warping: http://www.fly.elise-ng.net/index.php/i ... splaylite2 (I believe this directly interfaces with d3dx9.dll)

@OmniAtlas: There are a couple of developers on this site that are working on drivers to convert preexisting games to the Rift format. One of them is open source. I would check there.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15086

http://www.mtbs3d.com/phpBB/viewtopic.php?f=26&t=14970

A few minutes with Doom 3 on the Rift

As a lot of you know, Palmer and company were able to demo the E3 version of Doom 3 BFG for the public with the Rift on Saturday at QuakeCon. The short version is that it is awesome and demoed quite well. The hall opened at 9:00am, and by 11:30am the line was already an hour long. Occasionally people who were coming off the demo station would shout back over to the rest of the line, "it's worth it!" I had to wonder how many of them realized that some of us had been standing in that line for twenty years.

First up, some thoughts on the hardware.

They were actually using JohnCarmack's modification of the Rift (as seen in his E3 interviews with the blinders, black tape on the lens edges, and single elastic ski goggles band to hold it to your head). I was actually pretty happy to see that since it closely resembles the developer kit design. It felt reassuringly secure and lightweight. A few of my friends complained that it sat just a little too close (they felt their eyelashes brushing the plastic lenses), but I didn't have that problem. Lack of ventilation could be an issue for long-term use, but the blackout effect is really, really effective. Time to start researching DIY ways to allow air in, but not light?

I'm near-sighted and didn't have any trouble getting a focused image, but for my other friend (that requires glasses for pretty much everything), it was basically unusable. The glasses question was on everyone's mind at QuakeCon and I can understand Palmer's hesitation to create much more space between the eye and the lens; to do so would diminish the FOV pretty quickly. Larger lenses could fix that, but then you're in the game of trying to source new parts. Palmer has pitched a few ideas in interviews, so we'll see what they ultimately settle on there.

The display panel is probably the weakest part of the package. Resolution is on everyone's mind, so I'll get that out of the way first: it honestly wasn't a big deal. Yes, the grid and sub-pixels are there when you look for them. Yes, I will happily trade up when a higher-resolution panel is available. As it was, though, the grid was gone whenever I became engaged. My wish is for a panel with better contrast and a faster response time. Those two issues combined flattened out the feeling of depth somewhat, and more than the resolution, reminded me that I was in a simulation. Happily, the panel is also the part with time (and the massive arms race of the mobile industry) on its side. The panel that's there should be perfectly adequate for a developer kit, and even a heck of a good time in Doom 3. In the mean time, I would just author content on a traditional monitor and trust that the display in consumer HMDs will be up to spec.

Playing Doom 3 itself was great fun. Even though my friends and I knew that there wasn't any positional tracking, we still couldn't help dodge to the side in our chair a little bit as a fireball arced toward us. The segment on demo was from a very action-heavy part of the game when you have a drone helping you take down the constant stream of imps. It's a smart choice since it's relatively hard to die, and the player has some time to get oriented and be a bit of a tourist. Having a game pad input mapped to handling large turning worked well for seated play, as did decoupling the pitch of the weapon from the view.

I'm used to seeing VR demos with terrible graphics, so playing a polished game was actually a real treat. The Rift and Doom 3 do enough things noticeably better than anything else at the consumer level that I felt like I was experiencing a lot of "firsts" all over again.

Some other random impressions:

• The increase in vertical FOV feels almost more important than the increased horizontal in terms of making me feel grounded in a world. It practically filled my eye's entire FOV for looking straight ahead.
• There was a slight amount of perceivable latency in Doom 3. Overall, it's really really good. But I just want to set some expectations.
• Smoke planes, light-bending teleportation, and other "2D" effects stood out, of course.
• The 3D effect is really pronounced. Not in an artificial way, mind you. Just in a "I need to consciously not try reach out to grab that steam valve" way. I try not to give in to hyperbole, but I actually did have that moment with the Rage demo.
• Good VR is addictive.

Thanks for the feedback guys.

I'm prepared to get contacts to give this thing a good go!

Thanks Bill. Did you notice a difference in tracking responsiveness between the Rage demo and Doom demo? Carmack talked about doing some sensor forwarding in the rendering pipeline to reduce the delay. I don't know whether that was present in the Rage demo, and I was curious whether you noticed anything.

Edit:


I had the same impression, and I think I tried to grab that same pipe.

I absolutely noticed a difference between the two. Doom felt better, but I'm pretty sure it also was running on a better PC at a higher, more consistent frame rate than the Rage snippet.

@BillRoeske: Thanks for the reassuring words, though I was completely sold a long time ago...

I'm cross-eyed, about 20 degrees. Do you think there would be enough adjustability to get the two images into my field? For an example, most binoculars won't go close enough together for me. Am I right in understanding that at least some of the adjustment in the horizontal is done in software? Is the hardware in the present user-assembled model potentially adjustable by a someone who likes to play with optics? --- Thanks for this great, nuts and bolts discussion.

I think the kinds of lenses they are using allow for a range of eye positions.
Software-wise, you probably want the separation/convergence adjusted so that the centre of each eye's display is closer to your nose (off-axis rendering). I'm not sure if you'd want the cameras to be angled in or not.

Thanks for the reply. I hope a convention comes to N Calif in the next few months. I want to try one.

What an amazing read...

FInally I climb out from under my rock.. nearly 20 years have past since I donned a VFX1.

This time they have my full attention ... I am on my knees praying this will usher in the greatly awaited age of VR.

I think I hit the BUY NOW button , after only three minutes of reading over the Occulus Rift site.

This will be the greatest Christmas ever !

Thanks Palmer & John for finally lifting the game