Head mounted display theory

[JDuncan]

This is a theory on how a head mounted display can show stereoscopic images and a large field of view.

Starting with the stereoscopic first.
The image being shown to the eye, is seen by the eyes at two different horizontal perspectives.

If you hold a pencil vertically and in front of you, your eyes are cross eyed as you focus on the vertical pencil.
Then move the pencil to the left or right until only one eye sees the pencil.
The pencil is now not visible to the two eyes but it is visible in your field of view.
So by only the field of view seeing it but not two eyes, the image stops being stereoscopic.

So there is a part of the display or image being shown to the eye that is supposed to be stereoscopic,
and part that is only the field of view and not stereoscopic.

As you may or may not know, the Oculus VR is using only the one stereoscopic position, focusing at infinity.
But true stereoscopic images has positive, negative, and zero parallax; three types of stereoscopic images.

My solution is to wear shutter glasses over the eyes, then have the LCD screen behind the shutter glasses.
Then when the left eye shutter closes, the display shows one image,
and then the right eye shutter closes and the left eye shutter opens and the display shows a new picture.
The pictures shown to the eyes are stereoscopic pictures, not just focusing at infinity.
Part of the stereoscopic picture the eye sees is being fused in the brain so it's stereoscopic,
Part of the picture belongs only to the field of view.

Because the glasses are worn,
I suspect a custom type of shutter glasses that lets the field of view be non-occluded be visible would be necessary.
So the glasses would look like they belong to a clown.

And since the glasses extend about a inch past the eye brow, a large monitor would be necessary,
like a iPad mini screen.

I'll draw up some pictures, but I warn you, I don't draw very well.



With the HMD now able to show 3D stereoscopic images, there can be eye tracking.
Eye tracking is so if the eye looks in virtual reality, it sees that in stereoscopic 3D, the far and close distance is in 3D. Things you see, can move in the three different types of stereoscopic parallax in virtual reality.

I won't build this myself, I would need to be a hardware genius to rig up some 3D shutter glasses into something like the Oculus which uses a ipad mini screen, and that's not even mentioning they're odd use of glass.
If of the shelf 3D TV shutter glasses are used, which initially for testing purposes it will have to be, then the field of view will be reduced.

[WiredEarp]

Actually, I think the Headplay HMD already used a single screen with a LCD shutter glass type filter, to direct the appropriate imagery to each eye.

The eye tracking and adjustable focus bit would be much more difficult to implement however.

[JDuncan]

I looked at the headplay video where the person takes it apart, and that is a very simple display.
https://www.youtube.com/watch?v=Q-TtFqasAkU

I think the oculus vr takes the screen of a cell phone display and uses leep optics to enlarge the half screen for the one eye.

But this means the eye needs to see the leep optics show that half of the display, which means the eye are looking straight ahead.
Cross eyed stereoscopic to see positive parallax isn't done in the oculus vr if the eyes are always focused at infinity.

I said that to show the eyes need to look at the other eyes viewing space when the eyes go cross eyes to see in 3D.
So the display space is shared between the two eyes.

Now with this is mind, the display is seen by the two eyes you have to show the image on the display in stereoscopic form to each eye.
So the eyes have some sort of glasses that lets the eye see the display differently than the other eye.

Then as the eye sees the display, the part of the picture not in 3D is the fov that lets the picture seem more immersive. For this the 3D glasses need to be deformed from regular glasses shape so the sides of the glasses are stretched out to the side by a few inches to maybe half a foot.

So the glasses let the eyes see the display in 3D and the glasses let there be a large fov. So the logical conclusion to these requirements is the display is in front of the 3D glasses and is a large size to fill the stereoscopic space and the fov space the eyes can see.
A display maybe the size of the iPad mini.
The display must be 120 Hz at least and able to show a 3D image to 3D glasses.

So if a smaller display could be used for this the leep optics the oculus vr uses maybe able to be used in this as well? I don't know.

[Fredz]

I think the oculus vr takes the screen of a cell phone display and uses leep optics to enlarge the half screen for the one eye.

The Oculus Rift is using a tablet display (Nexus 7 1st gen) and doesn't use LEEP optics but a single aspherical lens per eye. The HD Prototype is using a smartphone display though, most probably the one from the LG Optimus G Pro.

[HeliumPhoenix]

Have to go with WiredEarp on this one. Focal distance adjustment to simulate the changes in distance is outside the scope of all but the highest-end experiemental HMDs. You'd need fluid lenses, and the ability to adjust them quickly to match the detected focus change of the eye. The image source would also have to implement Depth of Focus blurring of each rendered frame.

Remember, most HMD optics attempt to fix focus at long distances to avoid eyestrain resulting from extended use. And since adjustable fluid lenses are very experimental (and expensive) using them to make the stereoscopic effect more 'realistic' isn't cost effective. Such a HMD would cost hundreds of thousands of dollars.......

[JDuncan]

To get the varying distance the object is perceived I would use stereoscopic images parallax.

First make binocular fingers and look through the holes your fingers make.
Then look at your keyboard in front of you of some image you can see with both eyes.
That image you see with both eyes is in 3D, no that 3D has a parallax value.
The SW uses the pupils distance from each other to decide what 3D images you can see and only shows you these in 3D.

Now if you increase or decrease the space of the holes in-between the eyes you can find the image your looking at. I mean if your looking at your keyboard, and you increase the space of the holes your eyes are looking through, you can only see one key on the keyboard in 3D or stereoscopically, but look up and into the distance and you can see a lot in 3D not just one thing.
But if you look up from the keyboard into the distance, AND increase the space of the holes in-between the eyes, as you look into the distance you will only see one thing in stereoscopic 3D.

So there is a path that shows the distance the holes are from each other, looking at what's closer to you the holes are closer together, looking at what's farther from you the holes are further apart. Spacing the holes reduces what you can see to only one image.

If this hole spacing is in SW and not with real holes, the SW uses the holes to decide what image to show to the eyes.
So what if the HW and SW worked together so the SW knows - at what distance the eyes see the image in 3D, the holes need to be this far apart.

So the image is shown to the eye and the HW makes the holes small and spaced apart so the eyes only see that image, this is recorded in SW, and the for all the distances the eyes see the HW makes the holes spaced so far the image is visible in 3D. Now the SW can make the image in 3D.

You see the image this far away and the SW makes the parallax distance just far enough you see the object in 3D.

The 3D is shown on the screen and the 3D glasses separate the screen from the eyes and the SW decides the eyes can see the image using the right parallax value.
If the glasses sides are stretched out, the fov is increased without occlusion from the sides of the 3D glasses.

The glasses would have some sort of blinder available that could allow you to find the spacing of the eyes as it sees the image on the screen so the correct parallax could be found. These would be fancy 3D glasses or some tech layered on top of the glasses that can give the blinders effect that acts to space the holes the eyes see through.

[JDuncan]

I will reiterate what I just said and add how to track the eyes.

The parallax of the image is understood by this analogy.
There is a pencil held in front of your face and the pencil is in-between a TV, and what's being shown on the TV is a horizontal number line, so the eyes see the pencils background is the number line on the TV.

When the two eye focus on the pencil the number line is doubled somehow, there's two of the number zero for instance.
But when one eye is closed and looks at the pencil the number line is not doubled, there is only one zero on the number line.

When one eye looks at the pencil the background of the pencil is a number on the number line not zero. The closer the pencil is to the TV the small the number on the number line will be when one eye looks at the pencil - the farther away the pencil is from the TV the larger the number on the number line the pencil is over when one eye is closed.

Now if the pencils distance from the TV changes it is viewed by the eyes, and the eyes pupils have a distance for how close they are to each other.
When the eyes see the pencil over a number on the number line, it's because there is a eye looking at the pencil because the eyes are that far away from each other.
If the eyes are not that far apart then as the eyes look at the pencil they see it over a different part of the number line!

So there is two measurements that need to be done for 3D;
- the distance the pencil is from the TV so as it is seen by the single left and right eye it's over this part of the number line.
- the 3D parallax the TV is trying to show, so the eye sees the pencil over a number on the number line, actually meets the eye as it shows the picture.

So a video picture of a ball being thrown and caught in VR, the image of the ball flying to the glove and finally being caught, that is one frame of video the moment the ball is caught. And that frame of video has a parallax value that the ball has and this must meet the pupils of the two eyes. And the ability to meet the two pupils is based on how far apart the eyes are from each other.
So one persons eyes are 70mm apart and they see the ball get caught so the parallax must be made so the image shown meets the eyes at 70mm apart, and another persons eyes are 63mm apart so the image shown to the eyes of the instant the ball is caught must meet eyes spaced 63mm apart.

And that is how there is two types of parallax, one the TV or display shows and is based on generic pupil distance, and one that meets the pupils of the unique eye when it's shown from the TV or display.

To get this distance the pupils are from each other, you show a video and on the glasses you have vertical blinders that block a part of the image.
Then you space the blinders so it finds the pupil distance for each distance the eye can see.
When the blinders are on to see the image being shown the blinders have to be moved towards the bridge of the nose.
This shows that when the blinders are this far from the bridge of the nose the object being shown on the screen is visible, and so all the parallax values can be found and stored in SW for later use, because the blinders at one parallax are not the same for a different parallax.

So that is part one showing the image, now to have eye tracking.

I think if there is a dot on the bridge of the nose where the skin doesn't move as the face changes it's shape, this can be the static position to give reference to the moving positions.

And the moving positions of the eyes are colored contact lenses.
A camera can see the static dot on the bridge of the nose and the colored contact lens of the eye, and pass this onto a SW that can find the difference between the still and moving colored parts.

Then as the blinders are used to find the distance the eyes are as it sees the image so the image can be shown to the eyes and the eyes see a 3D image, the camera sens the position of the contact lens to SW and this uses both eyes position relative to the still colored dot on the bridge of the nose.

Then as the eyes look around the video it looks at one part of the picture, and as the eyes look at that part of the picture the eyes change it's position relative to the still colored dot, and this shows the parallax of the eyes.
So the eyes look at this part of the picture and have this distance so that SW shows the eyes a picture with the correct parallax value so what they are looking at is in 3D.

Now how do you find what part of the picture they are looking at?
I mean you don't know just be measuring the distance the eyes are from each other what the eyes are looking at.

So the video has a grid that has lights for each grid, and the eyes look at the grid light up. So the eyes looking at the grid light up makes the contact lens that far away from the still colored dot, and that is the value for what the eye is looking at.

[JDuncan]

In other words,
if you send a video of the ball being thrown and caught.
The parallax is being made to one distance of the pupils.
And so the 3D seen is the object is one size.

But then you need to adjust for different pupils distance from each other which changes the parallax, and so the image size looks smaller or bigger.

So the smaller person sees a smaller world than a bigger person.
If you sit a a large and small person down and put a circle say a orange before them at mm from their face.
The bigger person will see a different sized orange than the smaller person.

Once this is able to be corrected in SW, so the bigger person sees the image differently than the smaller person.
You can worry about parallax perception which is based on the distance the pupils are from each other, and eye tracking the eyes look at this part of the screen and have this pupil form meaning they are looking at this parallax.

So first things first.

[JDuncan]

I may be wrong, but here is a way to create a 3d tv.

Make the source video a bunch if two dimensional layers.
So you have picture that shows only a part of the entire picture;
and by stacking these layers you see the entire picture.

For instance, if the clothes on the person were one layer,
the person under the clothes is another layer,
and the world the person is in is another layer.

And now you stack all three layers on top of each other to see the entire picture.

Now if you show a picture is has to have some light bounce off.
Like a light bulb hitting the surface, the light hits the top of the surface being shown to the person to brighten the image.

Now the image is being shown on see through glass and the image is dimly shown.
But when the light hits the image from the front the image becomes clearer and more colorful.

So the technology is to
- first cut the image into slices so the one frame has multiple layers that stack on top of each other to show the one frame.
- then show the layer using see through panel but the image is faintly visible on the panel.
- now illuminate the layer, not layers, each layer is lit individually, so the picture shown on the see through glass becomes visible.
- Then for all three layers or however many layers they all light the front of each layer and now the entire picture is shown and looks 3D.

- since you can guide reflective light, when you shine the light onto the surface of the screen the screen can be wavy, and there can be two lights that sequentially hit the screen at a high frame rate.

So when the light hits the screen it hits it at an angle so the reflected light beams onto the zone the right eye is at,
and likewise when the light stops hitting the surface of the screen the right eye stops seeing a picture, now the second light turns on and hits the screen and a different part of the wavy surface, and the light reflects to the other eye the left eye.
The lights for the left eye and right eye keep turning on and off so the picture is shown to the left eye then right eye so the picture shown to the eye is a stereoscopic picture.
Now the person sees 3D.

_________________

How is this stereoscopic TV idea relevant to VR headsets, you may be asking?

The screen in the VR headset has the see through screen that has a faint image shown on it.
The image has a light beam on it from the front and this lights up the image.

Now to get stereo the surface of the screen is wavy so the light beaming on it hits the surface at the angle the wavy surface of the display is showing the light source.
By having two light sources you can hit the surface at two opposing angles and the light as it hits the surface is then shown to the eye in the VR helmet.

Because the image on the screen is dimly shown, when the light hits the surface, the angle not being lit by light hitting it is only showing a dim image. So this dim image is cancelled in the brain as the persons focus goes to the lit image.

This is a way to have one screen like the oculus VR has, but still have 3D on the screen showing different parallax to the eyes.

So you wouldn't need shutter glasses or something passive 3D glasses this wavy surface and light and dim picture would be good enough to show 3D to the eyes of the viewer.

[JDuncan]

I know that introducing shutter into 3D creates an unnatural effect.
With shutter meaning left eye sees then right eye sees.

So I said to have a TV that has the pixel on a angle.
So the pixel is multiplied by 2, 1 pixel on each edge of the angle.

Then the pixels were angled so the pixel at the far end of the TV is visible to the eye in one spot.
This way the eye sees all the pixels of the TV.

Now the eye sees in stereo only one person sees in stereo 3D and only if their eyes are in a certain position.
This is not possible to do, so to fix this you need to move the screen, by tilting it like a seesaw teeter totter.

Now you have a seesaw teeter totter screen so the flat panel actually moves, you need head tracking.
Like in virtual reality, that kind of head tracking.

The head tracking shows the TV that your two eyes Interpupillary Distance measured, is at whatever position you move your head.
Because the pixels are static, the Interpupillary Distance will have to be compensated for by the viewer to sit far enough away from the TV to get the correct Interpupillary Distance .

Now The TV blinks the picture on and off.
This removes blur that is bad for 3D.

So four parts for auto-stereoscopic TV;
the pixels is multiplied and put on a angle
the screen tilts
there is head tracking to say the eyes are at this position.
the TV blinks the picture on and off.

Now when the head is at one position, the screen is tilting to show the picture to that position.
As long as the head tracking says that both eyes can't see the entire screens pixels, the TV does not blink on.
But when the head tracking says the eyes can see the pixels the TV blinks the pixels on.

Now x number of people can all see 3D if they sit far enough away from the TV so their Interpupillary Distance works for them based on how the angle of the TV sits the pixels.

This takes the blinking and head tracking idea of oculus vr, and the pixel idea I had, and merges them together.
The wobbling screen works like the avegant "glyph" headset, because the glyph tilts mirrors.

Now to get the correct Interpupillary Distance regardless of how far you sit from the TV, you would get rid of the pixels on a angle and use the glyph tilting mirrors.
When the mirrors tilted one way they tilt so the eye sees all the pixels but only one eye sees all the pixels.
Then based on ipd and head tracking, when the mirrors tilt to show the picture to the other eye the other eye sees all the pixels in their Interpupillary Distance at whatever distance the eye is from the TV.

Now if the glyph mirror tech is used instead of the pixels on a angle,
and the head tracking beams the picture into each eye sequentially,
you can theoretically move your head closer to the TV and still see 3D.

Now if you can move your head closer to the TV and still see 3d the picture being shown can take the head tracking and change the displayed image so you get a virtual reality or holographic effect when you move your head closer to the TV.

Imagine looking around a corner and looking down a street in virtual reality all by bending in your chair and moving your head in approximation to the TV.
Now as you look down the street which is around the corner you see agent smith walking towards you.
Now you move your head approximation to the TV sow the displayed image from the TV is no longer looking around the corner.
You get the eerie sense that something is around that corner, add in AI and you are in the matrix, literally by using holography and AI.
Now be nice to the agent.

http://www.youtube.com/watch?v=m-nPQQWIf3c

Now the Oculus vr uses the blinking tech and head tracking, so I decided if this tech uses oculus vr tech then by sharing it maybe I can help oculus vr tech because I have some other ideas mixed into that. I don't know what they are doing head tracking wise and stuff, so I just put this out there in case it might help in some way.

[JDuncan]

Look at this article from it;
http://newsoffice.mit.edu/2014/glasses- ... ector-0516

"
Over the past three years, researchers in the Camera Culture group at the MIT Media Lab have steadily refined a design for a glasses-free, multiperspective, 3-D video screen, which they hope could provide a cheaper, more practical alternative to holographic video in the short term.
"

They use the three layers to show a holographic picture. Now to get a true holographic picture there needs to be something to let the person interact with the picture on the display.

The eyes at one position look at the TV which is in another position. Then the picture is shown to the eyes.

Now when the eyes move, the display reacts to the eyes new position.

If the eyes move down then the image shows them what's down, if the eyes look up then the TV shows them whats up, if the eyes look left or right the TV shows them what's left or right.

This way the different layers can show a holographic image using the layers technology, and the image looks virtual / holographic because what's seen on the TV reacts to the persons head position relative to the TV.

Now to get the layers to work under all positions the person looks at the TV regardless of if the person stands right next to the TV and so forth, the TV must rotate so the layers on the TV suit the eyes so the person sees a holographic image from the TV.

So idea 1 is the TV tracks the eyes and shows a image based on the eyes position from point A time to point B time.
Idea 2 is the TV is on a robotic arm that swivels the TV to match what the person does, if the person walks up to the TV and stands right by it the three layers will show a 3D image under all situations the person gets close to the TV.

By the eyes having interocular distance, or space between the eyes, and the TV having three layers, the two eyes see the three layers differently which shows 3D/holography.
Therefore since the picture looks 3D based on the eyes left right position, then more layers affects the parallax the eyes seen by the two eyes, and so the perceived depth of the 3D picture.

For instance, hold your hand out and flat so the pinky is facing out and the thumb is facing in or towards you.
Then rotate the hand slightly not all the way so you see a flat hand, just so you see the fingers on the hand. This is the effect of the one eye seeing parallax layers differently than the other eye which affects perceived 3D depth.
That is what stacking layers does to create 3D image, it works by the eyes interocular left eye from right eye distance.