My project might go in a commercial direction, but even if it does, I am going to make all the plans as available as I can. Even if I end up in the HMD business as a job, I will always make my plans available for DIYers! If someone really wanted to copy me, they could always just buy a unit and rip it apart to see how it works anyways. I will have the results of all my PR3 experimentation posted in a few months, with the pros/cons of each lens assembly I am experimenting with.

Here is the main problem with an open source HMD project: There is very little to research, and even less to develop. The PR2/PR3? They exist because of some amazing twist of chance that provided a high pixel density panel that is EXACTLY the correct size for a single panel HMD. If it had been 5mm wider or narrower, the performance would be severely impacted. I wish that there were other lucky finds like that out there, but in all the research I have done, I do not think they exist!

I mean, yes, a design like the older Virtuality HMDs would be ideal (Small screens, mirror, and simple optics), but where are we going to get those small displays? The market for high definition displays has been trending towards screens that are at least 4.3" diagonal, and those are all driven by cellphones. What we would need is a high definition panel, less than 3" diagonal, that can be driven with a standard video signal. Thus far, there is nothing like this in the sub-$3000 range, and there are no short term mass market applications that will drive the price down. Even in that case, if we use mirrors, we need drivers that support horizontal flip. IZ3D supported that, sure... But they are looking to be dead. As we move towards standards like HDMI 1.4, we are going to see LESS flexibility in 3D drivers, not more.

And that leads to software. The best thing that could happen to the DIY HMD scene right now would be a brand new 3D driver with lots of output options. The biggest one? Adjustable binocular overlap! Right now, 3D drivers only output 100% overlap imagery. If we could get around that, then we are a heck of a lot closer! I have some 3.2" 800x480 IPS LCDs that are driven via HDMI, and they would be perfect for a wide FOV HMD with about 80% binocular overlap.

In summary, an open source effort might be too early at this point, we do not have the hardware available. It would be like starting that open source race car project back when the only engine available was a $200,000, 1000lb steam engine, or that CNC mill project when the most advanced cutting tool available is a flint knife. It just has nowhere to go. If someone can find a sub-3" display with SVGA or higher resolution, then this whole game changes, but until then, we are just stuck waiting around, or using something like the PR3.

I know, pretty cynical, but that is my honest assessment.

(On a sidenote, an HMD wiki might not be a bad idea. I could also provide hosting, I have a Hostgator reseller account, and I have a console modification wiki from the main site I run that could be adapter very easily)

A question that comes to mind with your above comments Palmertech (and I mean this purely from an open conversation perspective), is whether your time investment in the PR-HMD's and research might be coloring your ability to see other potential pathways for the hardware (and or what's viable). You are set on your PR3 design (as you should be) but that does not mean it's the only viable way to build an HMD with a wide FOV and high resolution.

When it comes to software, in many ways I think you've got it backwards. If you're trying to build an HMD with commercial viability you should be designing hardware that plays nice with popularly supported software standards. Much in the same way that Sony's HMZ-T1 works with Nvidia 3D drivers. From that perspective, the hardware has to reflect what is technically achievable with commercially available driver sets and supported games. In other words, you're going for the highest common denominator here and figuring out what form factor follows.

I suppose that in many ways tells us that perhaps the HMZ-T1 is the only real viable option right now in terms of cost, hardware and compatibility but I know and you know that Sony isn't pushing the hardware as far as they could. They could easily go for a 6 panel display (3 per eye) wide field optics and head tracking. In fact, that unit is sooooo close to the mark that its incredibly annoying that they didn't go the extra mile there.

If I really were focusing on the PR3, then yes, it might be coloring my reasoning. The thing is, though, I am terrible at focus. The PR2/3 actually got finished, but I have several other projects going that I would be glad to post. One of them relies on Kopin microdisplays (47 degree FOV), the other relies on SVGA LCOS panels (35 FOV), and another relies on high magnification beamsplitters paired with the aforementioned 3.2" displays! I am also working on a project that uses two of the panels used in the PR3 to create a 120 FOV, 720p per eye HMD. Problem is, it is ending up far too large, so it might have to be a boom mounted display. On top of all those, my personal projects, I am also working on a few HMD concepts at my work! I won't talk about them, except to say that the cost of building those units is in the $$,$$$ range.

And yes, that is certainly the way to go about real commercial success. Problem is, working with those standards takes quite a bit of money! First you need to design a control board for the panels that can turn HDMI 1.4 into distinct outputs, and then you need to get HDCP, HDMI, and possibly Nvidia 3D Vision licensing. Going that route would mean starting a legitimate company, looking for venture capital, and taking a huge leap of faith. Granted, I would love to do that... But I think I would have a way better shot in 5 years. And besides, that is getting way out of the realm of DIY.

Like I said, if anybody knows of a sub-3" panel with decent resolution, then we are all set for making awesome HMDs. As a DIY community, we need either new developments in displays that let us use existing drivers, or new developments in drivers that let us use existing displays. One avenue that I have not experimented with much is projection based HMDs, those could possiby be a viable solution. But again, there are no drivers that would support them other than IZ3D, which is dead.

Sorry for rambling on and on, guys, I am running on very little sleep.

@ Bishop
No way could sony make a 6 oled hmd. They are taking a risk with the hmz, if they made what you are talking about, it would cost at least $2000. At $800, it is for vr geeks, home theatre geeks, and an impulse buy for rich people. At 6 oled, $2000+, it is for rich vr geeks, no use for home theatre, and no longer an impulse buy for generic rich people. Even if they did build it, I don't think it would be much good. It would have super wide horizontal, but that would highlight the limited vertical fov. It would be like looking through a slit in a riot shield.

I sometimes think that people on these boards have a slightly warped view of what things cost. At least in the UK, $800 is more than what most people spend on a tv for the whole family, never mind a hmd for one person. A 32" lcd is a common choice, which is about $400 over here....

@Palmertech
"Even if I end up in the HMD business as a job, I will always make my plans available for DIYers!"
I have a beer here for you, but I can't figure out to attach it to this message. I will drink it for you, I promise to enjoy it


Why is iz3d dead? As long as they don't pull the same trick as some other company that I won't mention and deliberatly remove support, I don't see how they can be dead. Stopped improving perhaps, but even with all it's glitchs and crashes I still think it is ok. Even if it never really gets any better, as long as they keep it working with the next version of windows/directx, and don't start going backwards, it is good. Or have I missed something?


Slightly off topic, but I find lack of sleep can sometimes be good.
I can program a little, but I'm not great at it. Lack of sleep significantly improves my programming ability. I don't really know how to describe it, its like when I'm too tired to think, the little bit of my brain that understands coding has direct control of my fingers, I spew out a big pile of code, that is glitchy but mostly works. Of course the problem with that is when I'm less tired, I can't understand what I typed so I cant fix the bugs!

Ignoring any additional information I might have, let us examine the public information on IZ3D we have:

1) No posts by their own staff on their forums, multiple people there who need license activations are not getting them.
2) Last driver release is a beta from early June with many notable bugs, unfixed for months with no estimated date of completion.
3) Vague early August blog post on their website that says they are switching business direction, whatever that means. No updates since.
4) Cancellation of their new display, then their new universal shutterglasses that were supposed to replace it.
5) No new devices licensing the IZ3D driver in the past year.
6) Not even selling replacement glasses for the IZ3D monitor anymore.
7) Ignoring all messages via email, PM, and Facebook

Considering the implications of all that freely available info, it seems unlikely that IZ3D is in a position to improve their driver. For all we know, the next version of DirectX or Windows breaks it (Like DX10 and 64bit initially did), and that is the end.

You can find 3" diagonal displays with reasonable resolutions in cameras, if 3" is too big I guess lower size displays could also be found. For example the Nikon Coolpix P7000 incorporates a 3" TFT display of 921,000 dots (most certainly 1280x720) and it can be found for $299 online. Also maybe a design for 3,5" displays can be created considering this size is used in some cell phones (iPhones for example). After all one of the first HMD (VIVED in 1985) was based on commodity hardware like this, using Watchman TVs from Sony at the time.

What do you mean by overlap exactly ? I don't really visualize how it would work.

Also I think an open source stereo 3D driver is something feasible, even if it's not an easy thing to implement. With some skilled and motivated developers I'm pretty sure a working proof of concept could be implemented quite rapidely.

Yes, I also do think that a Wiki summarizing all the ideas and constraints evoqued here could be quite useful. Even if only to tell what is not possible at this time or what type of hardware would be needed in the future.

@Palmertech
Fair enough. I didn't know any of that.

I read that the new direct x is heading towards including stereo vision. This would make 3d drivers un nescescary. I'm going away to sweep up the dreams you stomped

Well, for what its worth, I hope to have my own 3D driver available at some point. The main idea would be to support unorthodox devices (displays, controllers, etc.) that other companies have abandoned or don't see as viable. So, yes, you will be able to adjust the stereo overlap and all things like that. I don't want to hype it up too much, since I am a busy guy, but this is going to be a focus of mine for the upcoming months. Realistically I want to have something ready in early 2012.

I tend to agree the big issue is the panels.

I'd suggest the place to start would be with iPhone panels the old 3G panels are dirt cheap, and the retina display will start to turn up in quantity. They are not an ideal form factor, but there are several ways they could be configured to work. The issue is how to drive them?
Wifi is doable, but even ignoring the weight issue I hate the concept of having to have 2 entire Ipods in the display.
What we really need is a way to drive them with a VGA or similar signal, and that's likely not simple.

I'd stay away from the driver issue initially, personally I'm more interested in demonstrating what's viable, then solving the more general problem of playing existing games on it.
I think you'd find many games would be virtually impossible to play on a 100+ degree FOV display because of HUD layout issues among other things. You'd really want to tailor experiences to the device.
I think simple demo software would validate it's existence, at least for me. Hey if you could make it cheap enough and people actually built them, convincing a Tridef or an iZ3d to add support may end up being viable.

If not I suspect adding a post pass to existing drivers to allow image shifting/masking and flipping wouldn't be insurmountable.

I've played with pico projectors and as is they don't lend themselves to usable configurations, some of this is correctable with more/different optics, but I'm not sure how viable this ends up being.

All existing drivers assume that the left eye and right eye fields of view overlap 100%.
If you place two reasonably sized displays in front of the eyes with optics, you will need for this not to be the case or you will simply not be able to converge the images.
Think of it as adjusting the images to account for accommodation rather than adding a prism to each eye.

Do you mean something like that ? :



Wouldn't this mean that HMDs don't have the same repartition of HFOV for each eye (ie. not symetric) ? Wouldn't only the HMDs with a HFOV > 120° be concerned by this, since 60° is the limit of the inward visual field for each eye because of the nose ?

If you didn't start programming that yet and want some starting point, you may have a look at this open source project which is basically a stereo 3D driver supporting OpenGL and anaglyph output :
http://sourceforge.net/projects/ilstereo/

Could be useful to learn DLL injection, which should be one of the most difficult part for a stereo driver under Windows.

@Fredz: Ok thanks. That might help. I'm just in the research stages right now.

921,000 dots is not much at all, certainly not 720p. That is counting all the subpixels! 640x480x3=921,600 dots (That is how Vuzix named the VR920, it had about 920,000 dots). So unfortunately, that 3" display in the P7000 is the same/lower res than you can get from the PR3. Even if it was higher resolution, we still have no way to drive the panel.

And yes, a design for 3.5" panels could be made, but we would need to have drivers that support adjustable overlap. The picture you made is somewhere correct, but you made a mistake: See how you made the squares a lot bigger in the image with overlap? Don't do that! If you make them the same size, you will see that you get a lot of extra space on the far right of the right display, and the far left of the left display; This lets you use panels that are too wide to place directly in front of each eye, and also greatly expands horizontal FOV.

@ERP: iPhone 4 retina displays, why not. But like you said, driving them is hard. My background with displays/control boards actually far predates my work with HMDs, and I gotta tell you, it would be next to impossible to use iPhone 4 displays. These displays are made to be driven directly from SOC (System-on-a-chip) units that are used as the basis for most embedded devices, and these devices have digital video outputs designed specifically for those kind of displays. A matched pair. Now, if you could have a custom ADC (Analog to digital converter) microchip designed to drive them, then you are all set! Problem is, that costs tens/hundreds of thousands of dollars if you are not using a panel that has a common drive format (Like LVDS), and the panels used in these phones are of the former category.

Now, you could make a one-off control board to drive these panels using an FPGA (Field-programmable-gate-array), they are commonly used for prototyping things like video control boards. Problem is... You would have to reverse engineer the display (Proprietary cell phone displays are not very well documented, generally), and then configure the FPGA to drive it from scratch. On top of that, FPGAs are bulky and expensive ($,$$$+), especially ones that are powerful enough to drive high resolution panels. You can find some work where people used FPGAs to drive older iPod displays, the 320x240 ones, but doing the same with a retina display is an order of magnitude harder.

It is just a really, really tough problem. Sure, we could demonstrate what is "viable", but spending hundreds/thousands of dollars on something that only works with a limited software demo is not really worth it.

It is really rather depressing.

@ PalmerTech & Fredz: I just thought i'd mention that there are two different technologies that IMHO are promising for HMDs, due to their small size.

They are (of course) the OLED displays being produced by Sony, for the new gen of SLR type cameras that use digital viewfinders. I think Sony is the only one doing this so far but if they are popular (as I believe they will be) I suspect this will drive the adoption of higher resolution, small displays, whether OLED or LCD.

The other one is DLP chips. For example, one of these DLP chips is 0.55", XGA resolution. I see no real reason why this couldn't be used in a HMD, and there are LOTS of DLP devices out there currently so prices should be fairly reasonable. The biggest problem I can think of with DLP is the fact that most DLP systems use a spinning wheel to do the colour, which could create problems in a moving environment due to gyroscopic effects etc. However, I do think the concept is quite promising (am unaware of any HMDs that currently use DLP tho).

Problem is, those small displays are not useful for high FOV. I can definitely see Samsung getting into the OLED microdisplay business, but they would just want viewfinders, not immersive displays.

DLP is certainly promising, but the displays are usually even smaller than their transmissive counterparts. One of the HMDs I am working on uses a 0.5" SVGA LCOS panel, which operates in much the same way a DLP chip would. In addition, front-lit solutions like LCOS or DLP need a pretty long optical path, which makes squeezing high FOV out of them even harder (Just look at the Headplay!). The lighting system in my LCOS HMD uses a color-sequential RGB LED lighting scheme, so no moving parts to worry about.

Ah sorry, I thought this practise of lying about the resolution was only used for HMDs. 921,600 also equals 1280x720, hence my error...

Would the adjustable overlap be needed for 3" or less display panels too ?

This leaves the solution of an embedded HMD with the images directly generated by the phones like in this project : viewtopic.php?f=120&t=13822
But I can understand that it's not what most people here want and that it also brings other limitations.

No adjustable overlap would be needed for panels in the sub-3" range, because you could fit the two screens side by side and put the center of the screen directly in front of each eye. You could also use 3M stick on prisms to use 3.5" displays, but they are terrible quality. You can also use some prisms that Edmund Optics makes, but those can only adjust the angle 25 degrees max, not enough to use 3.5" displays.

It could be a good idea to ask a manufacturer to build prisms for the specific need of these 3,5" displays if an appropriate design is set, since this size seems to be quite common. Some people on a French website (hardware.fr IIRC) did this for a grouped command for anamorphic prisms and they did obtain something affordable (<$100) with optical coating and very good quality glass (BK7).

It could even be possible to create DIY prisms in resin or with glass/glycerin to validate the concept before just as they did. They even wrote a specific software to calculate the angles and the chromatic diffraction based on the dimensions and refractive index.

You could do that, but then you still need a way to drive those displays. And if you had a way to drive the displays, you could adjust the overlap in hardware, even!

One interesting path would be to make anamorphic prisms for the PR3 design. That is, have it squish the vertically stretched image of the side by side image down to the correct aspect ratio. You would get more pixels, and it would work with side by side sources, which are pretty common!

I just got the Lumagen Radiance Mini and there are 2 functions that are useful to your project. First is the auto conversion of any 1.4 3D format to the format you want. For example, you can input frame pack format and output as SBS. Second is the NLS = Non Linear Stretch. You can use the menu to define how you want to stretch the video format to fit your screen. You can modify the parameter for example to stretch more on the edge of the video compare to the center.

Well damn.

http://www.tmdisplay.com/news/2011/2011_1020.htm

2560 × 1600 6.1" LCD panel.

This would be so absurdly awesome. A little too wide, sure, but that can be fixed in software, or with something like the Lumagen Radiance Mini (Thanks for showing me that!)

You would get 1280x1600 per eye, or you could use a standard aspect ratio like 1280x1024. Still, this is absolutely crazy.

Palmer,

can you try the 2560 × 1600 6.1" LCD panel for use in the PR3? I can't read the info on that site, but how much does this display cost?

They have not said. Right now, it is purely a prototype, they do not even have engineering samples available.

Pity.

Here's the English link.

http://www.tmdisplay.com/english/news/2011/2011_1020.htm

Palmer - I don't know much about your wireless video transmitter, but do you think it can sustain this high of a resolution?

It definitely cannot. I mean, I suppose you could feed it a 1080p signal and scale it. With HMDs, the display resolution is a LOT more important than the input resolution, since you can see the pixels so much more easily than a normal display. For example, I would rather have a 320x240 signal displayed on an 800x600 HMD than a 640x480 signal displayed on a 640x480 HMD.

There are other ways, in theory. For example, current WQXGA monitors all require dual link DVI outputs to drive.

Here is a normal, single link DVI connector:


Here is a dual link:


In theory, you could run each set of DVI signals through its own wireless transmitter, and then you would have enough bandwidth. In practice, it entirely depends on how the control board for the panel works.

That's what I figured. It's unfortunate - but then again too little bandwidth is a nicer problem to have than too little resolution. I guess you could always go with the laptop-in-a-backpack solution and go wired.

"For example, I would rather have a 320x240 signal displayed on an 800x600 HMD than a 640x480 signal displayed on a 640x480 HMD."

Insanity! I guess I have missed something major, as that seems, well, insane. The point of higher resolution is to have more detail. What good is more resolution if adjacent pixels are being driven to the exact same colour because the signal is too low res? Perhaps the scaleing circuit perform some sort of anti-aliasing? Still seems odd.

You mentioned you are working on a new version of pr3. Will this have same fov? Never thought I would say this, but I think pr3 might actually be high enough. Too much more than 120 is going to run into problems with normal game software. With 3d hacks and tracking hacks already needed, would be good to avoid fov hacks.

A possibility for driving super high res panels is to have a laptop in a backpack, and have this receive the video over wifi. This way a low spec laptop can be used and never need to be upgraded. Instead of just using a gaming laptop, most of the computing can be done on a relatively cheap, relatively fast desktop machine. It would also provide a reasonable amount of low latency local processing. A decent programmer could come up with all sorts of interesting uses for that.

EDIT- We need to come up with a better term than "laptop in a backpack"

I think you are going to introduce an unacceptable amount of latency that way. You can only get away with it with the wireless video hardware because it's dedicated circuitry. I think if you start putting a general purpose CPU in the middle to do some type of video processing and relay it's going to be really laggy. Better to just take the "hit" and put a full-power gaming laptop in the backpack to cut out ALL the bandwidth and latency problems OR (gasp) lose the backpack and go wired.

That is normally the point of higher resolution, yes. But with the low resolutions that HMDs use, you can often see the individual pixels! It better to have smaller pixels that are harder to see, even at the cost of detail. And realistically, 1080p gives you enough detail where it is not a huge problem. Sure, more would be better, but that is not possible at the moment.

I am messing around with the PR3 lenses, I am not sure which direction to take. It might actually end up being closer to 80 FOV, if I can get a big jump in clarity/loss of weight that way. I have tried stretching the FOV past 120, and the main problem is that you waste so many pixels on the peripheral! Just not enough pixels.

And brantlew is right, a laptop that just streams video would be too high latency, especially for head tracking. To be honest, a high end gaming laptop would be one of the cheapest parts of the setup at this point, somewhere around $2,000.

You might be right. I wasn't planing on trying it, just putting the idea out there. However it is really a software issue. These machines are capable of imperceptibly small latency, they are after all what we always use to run the games/software. I have seen linux distros that claim to run realtime. This is obviously not true, but they might be close enough. The video processing will always take time, but a 2ghz cpu should be able to compete with a 500mhz asic.

I wonder what a custom fpga would cost? I would guess a few thousand $. But that is a one off cost. Once it has been designed any amount can be made for low cost. It is not beyond a hobby project, people have cloned Cray supercomputers on fpga just for fun.

"(gasp) lose the backpack and go wired."

NNNNNOOOOOOOOOOoooooooooooo!!!!!!!!!!

I would rather have less resolution. For stuff that you truly need the resolution, you would probably be sat down anyway, so better with a huge monitor/ one wall cave.



"I’ve spent the last year and a half or so constructing my own 1/10-scale, binary-compatible, cycle-accurate Cray-1. This project falls purely into the “because I can!” category"

http://chrisfenton.com/homebrew-cray-1a/



"I am messing around with the PR3 lenses, I am not sure which direction to take. It might actually end up being closer to 80 FOV, if I can get a big jump in clarity/loss of weight that way. I have tried stretching the FOV past 120, and the main problem is that you waste so many pixels on the peripheral! Just not enough pixels.

And brantlew is right, a laptop that just streams video would be too high latency, especially for head tracking. To be honest, a high end gaming laptop would be one of the cheapest parts of the setup at this point, somewhere around $2,000."

For your application 80 may be better, but I would prefer 120. If you don't want to work on that version anymore then that is your choice but I, and I'm sure others, are still interested.

I have to agree that it's your Project Palmer, but I was interested in the 120 degree FOV version. Perhaps with the Sony HMD system coming out we can find a way to get a mod that gives that system the 80 degree FOV with the advantages of 720p resolution.

A "laptop in a backpack" would fall under "wearable computing" so it could be called a "wearable computer" or "wearable PC". Probably more catchy is "wearable VR" which would indicate the further use of an HMD and wireless motion controllers.

With all the discourse I've seen about how important FOV is, I want to see it for myself - clarity be damned! I say go for 100+ FOV Palmer.

@bobv: We could always use the unimaginative acronym "LIB"

@Cybereality- If you know anyone that's working on a custom VR project, I'm selling my 2 complete MRG2.2 systems. I hate seeing them going to waste collecting dust in my office.

Should call it backtop as it no longer on our lap but at our back.

There is not really such a thing as a "custom FPGA". FPGAs are used to prototype actual dedicated components, so they are very flexible in configuration; Once you figure out how to program the FPGA to do something the way you want it, you move on to making the real ASIC (application-specific integrated circuit). And sure, people have cloned old supercomputers, but cheaper FPGAs like that are nowhere near capable of the kind of high bandwidth video processing we need. In the FPGA industry, units capable of what we need are marketed with terms like "Most flexibility in the sub-$20,000 range!"

And don't worry, I am still working on 120+ FOV units. Problem is, I am on a project with a deadline now, and I can't get the 120 FOV lenses light enough to fit the arbitrary weight limit (They want an HMD that is held in place with just a single elastic strap, like ski goggles). I am free to work on other things, but I need to have 4 units built by the end of November, so I don't have as much time as I would like.

I like high FOV, too.

@PalmerTech: Please, don't feel rushed. I'm sure most of us here have been waiting for real VR since the 90's, whats an extra couple months?

You don't need a laptop. You can use WHDI or WirelessHD. Would the Asus WAVI work?

@Cyber: I don't feel rushed, or at least any more than I want to be. I was just explaining the reason for working on a unit with 80 FOV, it is not by choice.

@Aphradonis: WHDI only has enough bandwidth for 1080, maybe 1200p. That display I linked to earlier was WQXGA (2560x1600), the WHDI standard does not hand enough bandwidth to support it, and it will probably be a long time before consumer demand drives them to do so.