How to build head tracking into a VR headset like the Oculus
Posted: Sun Jul 28, 2013 4:46 pm
This is a new approach to VR that builds in head tracking.
First I wll detail what 3D is, and then I will detail how to create a 3D image, then I will detail how to create a VR headset like the Oculus VR.
What is 3D?
Imagine a horizontal number line being displayed on a television.
You have the number 0 in the middle and then the negative numbers on the left and the positive numbers on the right.
Now sitting in front of the television and looking at the number line, hold a pencil in front of you.
Focus on the pencil and the number line will have double vision.
Because the number line will be double vision there will be two number 0's.
Put the pencil equally far from the two zero's on the number line when your focusing on the pencil.
This will put the pencil inbetween the two zero's in your peripheral vision.
Now hold your hand over one eye and still focus on the pencil, the pencil will be over one part of the number line not zero.
Then move the hand covering the eye to the other eye and cover that other eye, and the pencil still in focus the pencil will be over a different part of the number line.
The position of the pencil inbetween the two zero's is lost when one eye is covered, so the vision puts the pencil over a different part of the number line.
The two eyes seeing the pencil, not covering one eye anymore, and the 0 is double vision again and the pencil is inbetween the two zero's on the number line.
What the eyes see as it is focusing on the pencil with two eyes is the two images of the pencil over different parts of the number line.
The brain takes these two different images and joins them into one image so the pencil looks as if it's inbetween two zero's.
This is the principle of stereoscopic 3D.
By taking two different photos of something and showing the two photos to the eyes, the pencil between two zero's phenomena happens again so your brain joins the two different pictures into one picture.
When you get closer to the TV you hold the pencil inbetween the two zero's, you see two zero's because of the double vision from focusing on the pencil.
And then hold the hand over the eye and you see a different number, a smaller value on the number line.
This shows that tv size and distance affect how the stereoscopic pictures function.
If the stereoscopic images for the left and right eye were made for one TV size and to be viewed at a certain distance, then undoing this makes the stereoscopic phenomena invalid.
And does the 3D tv industry and movie industry have a standard on what tv size and viewing distance to use? No.
But this is a problem that games and VR headsets can fix, by building games to use the size and distance on the VR headset the stereoscopic images made should always be valid.
How to create 3D for use in a 3D VR headset?
Assuming the user taking the photograph is wearing glasses, the user is looking at the 3D TV, and the image on the TV is not in 3D yet, and the TV is showing the same number line as before.
The user holds a pencil inbetween the eyes and the number line and see's a double vision of zero.
Then he holds a hand over one eye and the eye sees the pencil over a different number on the number line, the first stereoscopic image.
Now the technology on the glasses is a swivelling laser pointer, and a camera is attached to the swivel laser pointer.
Where the laser points is where the camera films.
The camera has the same field of view as the person sees out of the glasses.
The laser of the eye not covered by the hand, points to the number on the number line the persons eye sees the pencil is over when they cover one eye.
Then the person agreeing the laser is true, the camera takes a photo, then the eye is covered by the hand and the other eye sees and then the process of taking a photo repeats this process and the second stereoscopic photo is taken.
Now the two photos are run through 3D photo software to be viewable on the 3D TV in 3D mode, and the lasers and camera are turned off and the glasses become 3D glasses and the person looks at the 3D image the camera took of the pencil on the number line and agrees if it looks 3D or not when they focus on the pencil.
The moving pencil
Ignoring VR tracking for now.
If the glasses taking a picture of the TV showing the number line takes the two photos and the person looks at the 3D image and agrees the 3D looks fine.
Then the glasses have a camera that is able to view the persons eye, one camera per eye.
The cameras looking at the eyes takes a photo of the physical position of the eyes when the photos of the pencil are taken.
Now this process of photgraphing the pencil is retaken but the pencil is closer to the TV.
And this happens for all distances the pencil can be inbetween the tv and the glasses.
These values are plugged into the virtual environment.
Then inside VR, the person can sit still and view the pencil inbetween the glasses and TV and then the pencil be moved towards and away from the TV, and the person still sees it in 3D.
The same as if the photos of the pencil moving towards and away from the TV were taken in real life and then were shown in 3D mode on the 3D TV and the person seeing if the pencil looked as if it were 3D.
How to create VR headset tracking part 1
Because the pencils position on the number line changes if two eyes focus on the pencil or not,
there is an 'x' shape from one eye then the other eye intersecting past the pencil.
That x shape is made visible by two intersecting lines drawn by the two lasers on the glasses.
With the pencil inbetween the glasses and tv held one distance from the glasses.
When the person moves closer to the tv the lasers behind the pencil become shorter.
When they move farther from the tv the lasers behind the pencil become larger.
When the lasers become shorter the pencil is closer to zero on the number line.
If this distance of the lasers is measured and virtually recreated in VR.
Then as the person moves the laser closer to the virtual TV, the lasers behind the pencil become shorter.
This recreation is equaling the virtual environment to the real world environment.
The shorter laser has a trackable value.
The virtual and real environments agree on the length of the laser behind the pencil.
As the pencil is one distance from the glasses but a variable distance away from the tv, the lasers on the number line give the value that can be plugged into VR.
If the person moves in a straight line towards and away from the tv in VR, the number line should show the laser on the number line change as much as it does in real life.
In the real world the persons shining the two lasers from the glasses onto the number line being displayed by the tv.
Then in VR this is recreated, so the virtual glasses are beaming the two lasers onto the number line the tv is displaying.
Then as the person in real life moves in a straight line towards the tv so the lasers change position on the number line,
then in virtual reality the lasers change position on the number line and the person is moving closer to the tv.
So the exact distance the person moves in reality that changes the number the two lasers are over is also mirrored in virtual reality.
How to create VR headset tracking part 2
Two different concepts;
The pencil is a static distance from the glasses and the person moves the distance from the pencil to the tv.
The person and tv are static values and the pencil changes it's distance from the tv and eyes at the same time.
How to create a VR headset tracking part 3
Now the person physically uses the static pencil virtual reality, but the virtual reality the person sees is the moving pencil and static tv and person.
So the eyes see stereoscopic images that look 3D, but the head position is measured in reality.
The VR headset in the real world must move as the person moves their head.
The person doesn't look at the pencil, but a pencil is used for the lasers.
So something like a unicorn horn is needed to put in front of the vr helmet to play the role of the pencil.
Then lasers on the vr headset beam onto the pencil creating the x shape I mentioned before.
The distance the lasers are behind the pencil lets the virtual and real world agree on measurment.
The left and right camera see the left and right lasers behind the pencil that is inbetween the glasses and the number line.
The cameras feed this into software which finds where the lasers are hitting on the number line.
Then the real world and virtual world put the persons head that far away from the number line,
so in the virtual and real world the lasers hit the number line that exact distance from behind the pencil that is in front of the persons glasses.
The person doesn't know what the lasers are touching on the number line, but only looking at what's in the virtual environment.
Now when the person turns their head in the virtual environment of the moving pencil,
the virtual environment using the number line is used to finds it's position.
Then the virtual environment of the static pencil is used by the virtual environment of the dynamic pencil to find the head position to enable head tracking.
This may mean a number line circling the person so as they turn the head the lasers focused on the pencil but still hit the number line.
Because the vr environment the person sees uses the eye position not the laser position,
the lasers can stay focused in one spot but the person still move their head around.
http://www.youtube.com/watch?v=oKAqvs4KTkk
click on picture to enlarge it
First I wll detail what 3D is, and then I will detail how to create a 3D image, then I will detail how to create a VR headset like the Oculus VR.
What is 3D?
Imagine a horizontal number line being displayed on a television.
You have the number 0 in the middle and then the negative numbers on the left and the positive numbers on the right.
Now sitting in front of the television and looking at the number line, hold a pencil in front of you.
Focus on the pencil and the number line will have double vision.
Because the number line will be double vision there will be two number 0's.
Put the pencil equally far from the two zero's on the number line when your focusing on the pencil.
This will put the pencil inbetween the two zero's in your peripheral vision.
Now hold your hand over one eye and still focus on the pencil, the pencil will be over one part of the number line not zero.
Then move the hand covering the eye to the other eye and cover that other eye, and the pencil still in focus the pencil will be over a different part of the number line.
The position of the pencil inbetween the two zero's is lost when one eye is covered, so the vision puts the pencil over a different part of the number line.
The two eyes seeing the pencil, not covering one eye anymore, and the 0 is double vision again and the pencil is inbetween the two zero's on the number line.
What the eyes see as it is focusing on the pencil with two eyes is the two images of the pencil over different parts of the number line.
The brain takes these two different images and joins them into one image so the pencil looks as if it's inbetween two zero's.
This is the principle of stereoscopic 3D.
By taking two different photos of something and showing the two photos to the eyes, the pencil between two zero's phenomena happens again so your brain joins the two different pictures into one picture.
When you get closer to the TV you hold the pencil inbetween the two zero's, you see two zero's because of the double vision from focusing on the pencil.
And then hold the hand over the eye and you see a different number, a smaller value on the number line.
This shows that tv size and distance affect how the stereoscopic pictures function.
If the stereoscopic images for the left and right eye were made for one TV size and to be viewed at a certain distance, then undoing this makes the stereoscopic phenomena invalid.
And does the 3D tv industry and movie industry have a standard on what tv size and viewing distance to use? No.
But this is a problem that games and VR headsets can fix, by building games to use the size and distance on the VR headset the stereoscopic images made should always be valid.
How to create 3D for use in a 3D VR headset?
Assuming the user taking the photograph is wearing glasses, the user is looking at the 3D TV, and the image on the TV is not in 3D yet, and the TV is showing the same number line as before.
The user holds a pencil inbetween the eyes and the number line and see's a double vision of zero.
Then he holds a hand over one eye and the eye sees the pencil over a different number on the number line, the first stereoscopic image.
Now the technology on the glasses is a swivelling laser pointer, and a camera is attached to the swivel laser pointer.
Where the laser points is where the camera films.
The camera has the same field of view as the person sees out of the glasses.
The laser of the eye not covered by the hand, points to the number on the number line the persons eye sees the pencil is over when they cover one eye.
Then the person agreeing the laser is true, the camera takes a photo, then the eye is covered by the hand and the other eye sees and then the process of taking a photo repeats this process and the second stereoscopic photo is taken.
Now the two photos are run through 3D photo software to be viewable on the 3D TV in 3D mode, and the lasers and camera are turned off and the glasses become 3D glasses and the person looks at the 3D image the camera took of the pencil on the number line and agrees if it looks 3D or not when they focus on the pencil.
The moving pencil
Ignoring VR tracking for now.
If the glasses taking a picture of the TV showing the number line takes the two photos and the person looks at the 3D image and agrees the 3D looks fine.
Then the glasses have a camera that is able to view the persons eye, one camera per eye.
The cameras looking at the eyes takes a photo of the physical position of the eyes when the photos of the pencil are taken.
Now this process of photgraphing the pencil is retaken but the pencil is closer to the TV.
And this happens for all distances the pencil can be inbetween the tv and the glasses.
These values are plugged into the virtual environment.
Then inside VR, the person can sit still and view the pencil inbetween the glasses and TV and then the pencil be moved towards and away from the TV, and the person still sees it in 3D.
The same as if the photos of the pencil moving towards and away from the TV were taken in real life and then were shown in 3D mode on the 3D TV and the person seeing if the pencil looked as if it were 3D.
How to create VR headset tracking part 1
Because the pencils position on the number line changes if two eyes focus on the pencil or not,
there is an 'x' shape from one eye then the other eye intersecting past the pencil.
That x shape is made visible by two intersecting lines drawn by the two lasers on the glasses.
With the pencil inbetween the glasses and tv held one distance from the glasses.
When the person moves closer to the tv the lasers behind the pencil become shorter.
When they move farther from the tv the lasers behind the pencil become larger.
When the lasers become shorter the pencil is closer to zero on the number line.
If this distance of the lasers is measured and virtually recreated in VR.
Then as the person moves the laser closer to the virtual TV, the lasers behind the pencil become shorter.
This recreation is equaling the virtual environment to the real world environment.
The shorter laser has a trackable value.
The virtual and real environments agree on the length of the laser behind the pencil.
As the pencil is one distance from the glasses but a variable distance away from the tv, the lasers on the number line give the value that can be plugged into VR.
If the person moves in a straight line towards and away from the tv in VR, the number line should show the laser on the number line change as much as it does in real life.
In the real world the persons shining the two lasers from the glasses onto the number line being displayed by the tv.
Then in VR this is recreated, so the virtual glasses are beaming the two lasers onto the number line the tv is displaying.
Then as the person in real life moves in a straight line towards the tv so the lasers change position on the number line,
then in virtual reality the lasers change position on the number line and the person is moving closer to the tv.
So the exact distance the person moves in reality that changes the number the two lasers are over is also mirrored in virtual reality.
How to create VR headset tracking part 2
Two different concepts;
The pencil is a static distance from the glasses and the person moves the distance from the pencil to the tv.
The person and tv are static values and the pencil changes it's distance from the tv and eyes at the same time.
How to create a VR headset tracking part 3
Now the person physically uses the static pencil virtual reality, but the virtual reality the person sees is the moving pencil and static tv and person.
So the eyes see stereoscopic images that look 3D, but the head position is measured in reality.
The VR headset in the real world must move as the person moves their head.
The person doesn't look at the pencil, but a pencil is used for the lasers.
So something like a unicorn horn is needed to put in front of the vr helmet to play the role of the pencil.
Then lasers on the vr headset beam onto the pencil creating the x shape I mentioned before.
The distance the lasers are behind the pencil lets the virtual and real world agree on measurment.
The left and right camera see the left and right lasers behind the pencil that is inbetween the glasses and the number line.
The cameras feed this into software which finds where the lasers are hitting on the number line.
Then the real world and virtual world put the persons head that far away from the number line,
so in the virtual and real world the lasers hit the number line that exact distance from behind the pencil that is in front of the persons glasses.
The person doesn't know what the lasers are touching on the number line, but only looking at what's in the virtual environment.
Now when the person turns their head in the virtual environment of the moving pencil,
the virtual environment using the number line is used to finds it's position.
Then the virtual environment of the static pencil is used by the virtual environment of the dynamic pencil to find the head position to enable head tracking.
This may mean a number line circling the person so as they turn the head the lasers focused on the pencil but still hit the number line.
Because the vr environment the person sees uses the eye position not the laser position,
the lasers can stay focused in one spot but the person still move their head around.
http://www.youtube.com/watch?v=oKAqvs4KTkk
click on picture to enlarge it