The Sony HMD is real!

Post by **3dvison** » Sat Mar 31, 2012 3:26 pm

cybereality,
Hope you will be doing one of your full reviews.

Post by **SDM** » Sat Mar 31, 2012 4:04 pm

@brantlew
Indeed, does bother me that I had to alter an expensive gadget like this to make it practical/wearable for me, was the reason I returned the first one. Really want the tech to succeed though, become more mainstream, there to be a "HMZ-T2" with flaws corrected, so decided to grab another if for no other reason then to try to help Sony/others to move the tech along.

@CyberVillain/mayaman/space123321
Should have taken pics while doing the mod. Thought occurred to me but didn't know if I'd be scrapping it all to redo, so unfortunately didn't take pics in the process. Camera battery dead now, but will try to show more detail on how things were done when I get the chance.

Did grab a pic of what was removed, can see below that the entire outer casing was removed. Thought briefly about keeping the two casing bits (like the guy on Youtube), but the nose slot gets in the way, it or the goggle frame would have to be cut partially away to use together. Wanted the goggle frame rigid, the HMZ case intact (in case I ever do sell it), so all of those case pieces had to go.

I did not weigh the bits removed, nor the final unit (can/will later though), but can see what's removed now in the pic below (minus all screws -except 3 original screws that were reused). Of course you have the weight of the goggles added back, a few little nuts/bolts and small aluminum plate, but the latter two of which are pretty much negligible. The aluminum plate I used was .020" I think, likely 6061-T6 scrap I had laying around (though doesn't need to be aircraft grade). Just kinda a triangle shaped piece (with forward facing tip cut off/flattened), 3 (approx 1/2") flanges bent up on the back side to drill/mount to visor. The plate hits the top middle/centered screw on the HMZ eyepiece assembly nearest your eyes, and the two screws ahead of that towards the top front (hence 3 original screws reused). Plate ends up covering the toothed gear bits for the slider mechanism. There's enough room to easily (and independently) adjust for IPD by just gripping each eyepiece using your thumbs on bottom of eyepiece casings, fingers on aluminum plate piece atop.

@cybereality
Thanks, be interested to see what mods you may do to yours.

-Steve

Post by **cirk2** » Sat Mar 31, 2012 5:28 pm

That mod looks so incebebly geeky, I like it.

Maybe you can mod it to look like this.
http://xboxoz360.files.wordpress.com/20 ... xcgn-1.jpg

I would like to try the HMD myselv, but Sony is on my personal blacklist.

Post by **cybereality** » Sat Mar 31, 2012 5:57 pm

@3dvison: Seemed kind of late to do a review, but if people want it I can whip something up.

@SDM: I'm a tad squeamish when it comes to modding $800 peripherals, but I may try some simple stuff that doesn't void the warranty.

Post by **3dvison** » Sat Mar 31, 2012 8:54 pm

cybereality,
You do a great job of reviewing and also the fact you have used other HMD's and Vuzix products makes you thoughts gold.
I also just wanted to see what words you would use, for the gap in visual quality between the HMZ-T1 and other HMD's ?..."Night and day" was a start..LOL

OH Sh*#*, I forgot, what do you think of Fuji W3 stuff show on the Sony cyber ?

Post by **pierreye** » Sun Apr 01, 2012 8:42 pm

@SDM - Cool mod. Can you take a photo on how you mount the internal of HMZ-T1 to the ski mask and the IPD adjustment knob? Since you removed the bottom pieces, how do you mount the HMZ-T1 control button to the ski mask?

@Cyber - Your opinion and review would be valuable as you had lot of experience with other HMD.

Post by **space123321** » Mon Apr 02, 2012 2:28 pm

@SDM

Man I am sooo tempted to give this mod a shot... Any chance you could take a few pics at different angles?... I still do not fully understand how the screens are secured into the goggles. Thanks in advance!

Post by **SDM** » Tue Apr 03, 2012 4:11 am

To those interested in how I modded my HMZ to goggles, I can't take more pics right now as my camera battery won't hold a charge (awaiting a couple new ones), but I did draw up a rough mockup of how I was going to do things in CAD before I actually finalized/performed the final mod. So, I decided to try making a video instead with the 3D rendering there to show how I did things. Actually is much clearer to see how things were done this way than via pics. Can find the video here: my mod video

Not a video editing expert, first video I've ever made lol, but hope it is clear. File is about 117Mb I think (sorry about that but wanted it clear with decent audio), never use video hosting sites so don't know how to host it on Youtube or such. Any way, f anyone downloads it, let me know there are any problems viewing it.

Hope it helps.

EDIT: I've done a second shorter video to add a bit more info on making things stouter for those who may need it, clarify a couple little points. Been up for about 24hrs straight now, was tired making the first video, beyond tired now, but think these two videos together cover everything.

Second video (only a few minutes long): HMZ mod part 2

-Steve

Post by **brantlew** » Tue Apr 03, 2012 6:41 am

Thanks SDM. Somebody should post this up to youtube for easy distribution.

Post by **CyberVillain** » Tue Apr 03, 2012 6:47 am

brantlew wrote:Thanks SDM. Somebody should post this up to youtube for easy distribution.

I can do it tonight if SMD allow me

Post by **SDM** » Tue Apr 03, 2012 7:57 am

CyberVillain wrote:
brantlew wrote:Thanks SDM. Somebody should post this up to youtube for easy distribution.
I can do it tonight if SMD allow me

Info is for everyone. If anyone can/wants to put my video on Youtube or such, perfectly OK with me.

Also, no big deal at all, happens a lot (especially since I tend to stick to electronic related forums were the SMD acronym is common of course), but I'm SDM rather than SMD. Probably should just change my initials lol.

Post by **CyberVillain** » Tue Apr 03, 2012 8:14 am

SDM wrote:
CyberVillain wrote:
brantlew wrote:Thanks SDM. Somebody should post this up to youtube for easy distribution.
I can do it tonight if SMD allow me
Info is for everyone. If anyone can/wants to put my video on Youtube or such, perfectly OK with me.

Also, no big deal at all, happens a lot (especially since I tend to stick to electronic related forums were the SMD acronym is common of course), but I'm SDM rather than SMD. Probably should just change my initials lol.

Sorry man, its the new HMD from SMD fault

Post by **SDM** » Tue Apr 03, 2012 9:42 am

CyberVillain wrote: Sorry man, its the new HMD from SMD fault

Ah, well that's better than being a surface mount device at least

.

I've done a second shorter video to add a bit more info on making things stouter for those who may need it, clarify a couple little points. Been up for about 24hrs straight now, was tired making the first video, beyond tired now, but think these two videos together cover everything.

Second video (only a few minutes long): HMZ mod part 2

Post by **CyberVillain** » Tue Apr 03, 2012 10:24 am

Thanks for your work SDM, im downloading as we speak, the speed isnt that great I'm located in Sweden, so it will take a while. I'll link here when its done

Post by **CyberVillain** » Tue Apr 03, 2012 10:44 am

Here we go, Will actually look at them later tonigh, So i do not hope you have yourself naked there or something else that can get me banned from youtube

[youtube-hd]http://www.youtube.com/watch?v=o3PP4vdq ... I6sTNwl23A[/youtube-hd]

[youtube-hd]http://www.youtube.com/watch?v=dhQeqJuE ... e=youtu.be[/youtube-hd]

Post by **cybereality** » Tue Apr 03, 2012 5:51 pm

Silly question: how do you get that mesh cover onto the headset? It goes on the forehead pad, right?

Post by **JohnCarmack** » Wed Apr 04, 2012 5:42 pm

At the start of the SDM HMZ Ski Goggle mod video, the connecter is called an "HDMI proprietary Sony cable". If this is actually the case, I would be Very Interested in the possibility of bypassing the main processing box and hopefully the associated latency that is being induced. While the ergonomic issues with the HMZ are well known, the more problematic aspect for me is that there is around 30 milliseconds of latency introduced between the video input signal and the OLED displays. If you have a very low latency control/display loop, it is painfully evident switching from a Z-800 to an HMZ-T1. The FOV and resolution are great, but the lag is a terrible limiting factor.

I would be interested in paying a bounty for the development of a bypass cable if someone wanted to take a shot at it.

In an ideal world, the two displays would look like a single 2560 x 720 display, but if it is HDMI I would guess that it just takes the standardized frame sequential 720P format.

John Carmack

Post by **bobv5** » Wed Apr 04, 2012 5:56 pm

Hi Doom guy.

If the box is introducing latency it must be doing something. What that is I can't say, but it will likely be a major job to avoid it. It would have to be a big bounty. Big enough that your best bet is to go to Sony directy. (I hope someone can prove me wrong)

EDIT- I think that is just what he called it for the video. If it is truly just hdmi, a bypass cable would be trivial. But if it is just hdmi, the control box makes no sense. Have a look at the PR2 thread. For game immersion it is a far supierior product.

Post by **JohnCarmack** » Wed Apr 04, 2012 6:35 pm

Lots of consumer devices unfortunately introduce a frame or two of latency for on screen menus and resolution scaling / format conversion. The OLED displays may well just export 1280x720, and the processing box does all the 1080P / side by side / top bottom conversions, emitting 1280x720 to the displays. That is my hope, at least...

John Carmack

Post by **PalmerTech** » Wed Apr 04, 2012 6:42 pm

JohnCarmack wrote:At the start of the SDM HMZ Ski Goggle mod video, the connecter is called an "HDMI proprietary Sony cable". If this is actually the case, I would be Very Interested in the possibility of bypassing the main processing box and hopefully the associated latency that is being induced. While the ergonomic issues with the HMZ are well known, the more problematic aspect for me is that there is around 30 milliseconds of latency introduced between the video input signal and the OLED displays. If you have a very low latency control/display loop, it is painfully evident switching from a Z-800 to an HMZ-T1. The FOV and resolution are great, but the lag is a terrible limiting factor.

I would be interested in paying a bounty for the development of a bypass cable if someone wanted to take a shot at it.

In an ideal world, the two displays would look like a single 2560 x 720 display, but if it is HDMI I would guess that it just takes the standardized frame sequential 720P format.

John Carmack

Hi John,

A bypass cable is very possible. The final stage before the proprietary Sony cables is actually a DisplayPort transmitter IC (STDP4028), and on the HMD side is a DisplayPort receiver (STDP4020), so the obvious conclusion would be that the Sony is using a modified DisplayPort link, probably with consolidated grounds. From there, it looks like the final signal conversion for the OLED panels is done by a Spartan-6 FPGA on the HMD board.

It would certainly be possible to reverse engineer that link, but it would be hard. The HMD board has no vias to the other side, so the only way to trace the pins from the cable to the IC would be to remove the IC from the board, something that is hard to reverse (These ball grid soldered parts are meant to go on once and never move). From there, you could use the datasheet of the IC to match up all the different DisplayPort lines to their corresponding pins in the cable. Even then, though, there could be complications. The Sony might be expecting some funky authentication signal or handshake, and we cannot be sure what format, exactly, the HMD is expecting.

Before going to all that trouble, a few things to try:

1) Have you tried changing the various image processing settings on the control box to see if they affect latency?
2) Have you compared latency between 3D and 2D modes?
3) Have you compared latency between different resolutions?

Let me know if those make any difference. If not, I might be able to help you with making a bypass cable.

Post by **brantlew** » Wed Apr 04, 2012 8:24 pm

@JohnCarmack: Great to have you aboard John! Always been curious if any game industry insiders lurked around here.

Post by **WiredEarp** » Wed Apr 04, 2012 11:05 pm

If the HMZ is introducing a 30ms lag, thats unfortunately very significant, and turns me off it. I mean, in VR you want to keep your control/display loop as short as possible, and 30ms is really quite a significant lag.

I wonder if the ST1080 will have the same issue? if not, then that would definitely tip me towards it. Funny that John says that the Z800 is much faster, since I believe the Z800 is actually only 30fps in 3D mode (its buffered to avoid flicker) - yet is still apparently better in terms of responsiveness. Sounds like Sony didn't consider this important part of VR displays...

Post by **CyberVillain** » Thu Apr 05, 2012 3:58 am

I've played some BF3 online with the glasses and I didnt notice any lag, I'm sure i would if there were some because im picky,, Will have to try some more fast fps gaming when my ski goggles mod is done

Post by **JohnCarmack** » Thu Apr 05, 2012 9:36 am

My high speed camera is at the Armadillo Aerospace shop, so I can't make objective measurements right now, but the latency seems to be the same regardless of mode and format. That is to be expected -- you usually have to go out of your way to make a fast path around the image feature modes that have been added. I sent some information to Sony about this through one of our developer channels a month ago, but there hasn't been any response.

The latency on the Sony is partially compensated by the fact that the OLEDs switch very fast compared to LCD displays, but you can clearly tell there is a delay if you mirror output onto a monitor and the HMD. The full weight of the image processing only shows up when you compare it against a CRT or another OLED display. A side-by-side with a Z800 is the best apples-to-apples comparison.

Most VR demos have 100+ milliseconds of motion-to-photons latency, so the resolution and FOV of the Sony still make it pretty compelling, but I believe the threshold for truly immersive VR is under 20 milliseconds, which is kind of hard to do with 30 milliseconds of buffering... I am going to be testing a 120 hz, completely unbuffered prototype display from Emagin this month, which I am extremely excited about.

John Carmack

Post by **donkaradiablo** » Thu Apr 05, 2012 10:15 am

Hi Mr Carmack,

Does your research include in-game support for separate head movement and aiming?
Is there any collaboration with the graphics card vendors to bring 3D stereo support to the latest (and near future) id software titles?

Looking for a "maybe" or a "no", not expecting any promises

Post by **SDM** » Thu Apr 05, 2012 12:48 pm

Probably shouldn't have said HDMI cable in the vid there, sorry for any confusion, my mind was only set on the physical mod, not the electronic side of things.

Lag is definitely noticeable to me, quite significant with the HMZ. Even moving the mouse on the desktop disorientates me a bit, takes me awhile to get used to it each time I use the HMZ vs my regular monitor. Do adjust enough to enjoy it, but still bothers me. Hope the ST1080 fares much better in that respect, but if there's a way to reduce it with the HMZ, I'd certainly be interested.

There's a possibility I may be able to get a PCB (actually the whole unit for spare parts) from a non-functioning HMZ (owner broke it, so not in warranty, he doesn't seem to care much about it at this point). If so, would be up to "sacrificing" that PCB to see what's possible concerning the link.

Post by **CyberVillain** » Thu Apr 05, 2012 1:26 pm

I've now stripped the HMZ

Some one had questions about the controls, they are actually grouped together on a seperate platform so it should be easy to mount it on the ski goggle

Post by **PalmerTech** » Thu Apr 05, 2012 1:28 pm

I have not particularly noticed any lag on my HMZ-T1, but to be fair, I have spent less than an hour actually using it, and not with head tracking. It came right apart so I could document and mod the hardware!

I am hoping the ST1080 has less input lag, because John is right, 30ms is way too much.

Look forward to hearing your impressions of the Emagin prototypes, John! I have been pretty impressed with the Emagin SXGA panels I have seen in a few different HMDs, but not their newer stuff. What kind of field of view will you be looking at? Wide field of view has, in my experience, been the single most important things for immersion (Assuming other factors like tracking latency and accuracy are within acceptable parameters).

Post by **3dvison** » Thu Apr 05, 2012 2:57 pm

John Carmack,
The thought of you working with HMD's, just puts a big smile on my face.
Whether your doing it for fun, work or both, nice to have you here.

Post by **CyberVillain** » Thu Apr 05, 2012 3:28 pm

Are you for real the real John Carmack? If you are I must thank you for bringing us the best shooters for the last 20 years, very cool to have you here! I was 14 years old in 1994 when I won the Swedish championship in DooM II

@SDM, Is the material in the ski goggles "glass" really sturdy enough for mounting the HMD in? I was thinking of replacing the glass with a aluminium sheet cut out in the same form as the orignal glass and then mount the HMD in that, spray it black to block out reflections

Post by **SDM** » Thu Apr 05, 2012 4:22 pm

CyberVillain wrote: @SDM, Is the material in the ski goggles "glass" really sturdy enough for mounting the HMD in?

On my goggles it certainly is. The resulting (about 1" deep in my case) part of the curved "glass" (guessing probably lexan) that the HMZ/plate mounts to is very stiff in the important vertical axis, is made even stronger all around when all is assembled. My goggles also have a thinner clear layer of plastic behind the lexan, but it is quite thin/negligible, wouldn't effect things if it was there or not on a different pair of goggles.

So, with the goggles I used at least, an aluminum visor swap would not be necessary at all, would be of zero benefit structurally/functionally. Only make the mod more difficult by trying to fit the aluminum in place of the existing lexan. You do actually want some flex (past the plate flanges) as well, to be sure the goggles fit properly, the lexan allows for this nicely.

Post by **bobv5** » Thu Apr 05, 2012 8:56 pm

I would like to point out that latency is not necessarily too big of a problem. For a fast fps, designed to be be played with high dpi rodents, it is a problem. For a slower paced experiance, it might not matter. For example, left 4 dead, if it was designed for vr. You are not superhuman. You are slow and clumsy, just like reality. 30ms is not such a problem. (This wont work for ninjas, but I imagine they are too busy to be playing games anyway)

EDIT- I don't have the skills to create such a thing, but I think the way to get intresting VR would be more like books than films. Have the user playing through a story, knowing things will get bad, but not when or how. As apposed to running around in a magical bullet-resistant vest, vast ammonts of ammunition, and virtual death being insignificant.

I know I'm getting a little off topic, but the hardware we have is usable. The software is poor.

Post by **cybereality** » Thu Apr 05, 2012 9:20 pm

Well, I have not noticed much lag on the HMZ. I can believe that its there, its just not enough to be a serious problem for me. For straight gaming and video purposes, it seems acceptable. Now if you are doing some high-end simulations for the gov or what-not, then maybe its not up to snuff. But for most gaming and garage VR purposes it seems OK to me. I have not used it with a head-tracker yet, and I think any lag would be more apparent in that situation.

Also, I just found out about this computer case that I think would go perfect with the HMZ. May have to make my new build with this one:

http://www.corsair.com/vengeance-gaming ... -case.html

Post by **PalmerTech** » Thu Apr 05, 2012 9:22 pm

bobv5 wrote:I don't have the skills to create such a thing, but I think the way to get intresting VR would be more like books than films. Have the user playing through a story, knowing things will get bad, but not when or how. As apposed to running around in a magical bullet-resistant vest, vast ammonts of ammunition, and virtual death being insignificant.

Then start a thread talking about better VR software, because I (And probably the other people here) have a lot of ideas for that!

30ms of latency is not just a "clumsy" thing. Sure, it is fine for reaction time, but too much lag makes you cybersick a lot faster, and even if not, things can feel "off". Add on computational and tracking latency, and it can get bad fast.

Post by **bobv5** » Thu Apr 05, 2012 9:45 pm

Obviousy, it is better not to have latency. But, I woud rather keep ownership of my internal organs, and have have a crappy HMD.

I will make such a thread, but not tonight. It is way past my bedtime.

Post by **cybereality** » Thu Apr 05, 2012 10:02 pm

bobv5 wrote: I will make such a thread, but not tonight. It is way past my bedtime.

Too late, I have made the thread already:

http://mtbs3d.com/phpBB/viewtopic.php?f ... 43&p=72273

Everyone, please join in the discussion of VR software.

Post by **pierreye** » Thu Apr 05, 2012 10:28 pm

So far I don't have any problem with the lag issue including using wii motion plus as head tracker. No disoriented feeling when playing battlefield 3 using head tracking to look around.

Post by **brantlew** » Thu Apr 05, 2012 11:16 pm

Well John is a known perfectionist when it comes to timing issues. Certainly for quick twitch gaming it is important and I'm not so sure the hardware is up to snuff when it comes to that style of game. (Don't forget the massive delays incurred with the motion tracking). I can't see playing something like Quake well with a VR setup. But as others have mentioned, there are other gaming styles and genres that would be more forgiving of lag and would lend themselves to VR better. But I guess that's what the alternate thread is all about....

Post by **JohnCarmack** » Fri Apr 06, 2012 4:54 pm

Regarding the bypass cable, I would hope that it would be possible to find a source for the connectors and make a little breakout adapter for it instead of pulling the connector off of one of the boards, but the signals might be to tight for it.

If nobody recognizes it as a standard part, I could probably get someone to run it down with a tweet.

I think everyone is going to be surprised how much of a difference extremely low latency makes for VR -- current systems are allowing you to use your head as a controller, but it certainly doesn't give the impression of viewing a stable world through goggles. Of course, consistently running a game at 120hz is a bit of a challenge, but I think the payoff will be there.

John Carmack

Post by **PalmerTech** » Fri Apr 06, 2012 5:17 pm

I don't recognize it as a standard part. It is very similar to the mini-DisplayPort connector that Apple uses, but with a different number of pins. If this is just a one-off deal, then you could always just splice into the Sony cable.

How low have you been able to get your loop, John? The lowest (measured) setup I have experienced is about 52ms from input to eye, and it is great, especially with velocity based movement prediction algorithms. I am not much of a programmer, but the people who work on that stuff are wizards!