Hmm, after some thought. In the Doom BFG demo a head+neck model was used to displace camera positions, with the head rotating around the neck and the eyes being on the front of the face. This feels like a hard feature to do when watching recorded material. If you project the video on a sphere and move the cameras accordingly, the video will start to look distorted right?
But with no camera displacement how will that affect immersion? The camera would instead rotate around a single point which I'm imagining would feel a bit weird. Or will it be different as wrap-around video is not stereoscopic, if that at all matters? Hmm for sure.
A good way of thinking about the problem is like this. If you had a display mannequin and replaced the head with a 360x180 camera, you shouldn't be able to look straight down into the neck, but only down on your chest as your head tilts forward. Should this be done by placing the entire camera further forward, as video cannot be shifted in space? I think it's an interesting problem to solve, if possible.
EdZ wrote:The 'standard' method in gaming is called a Cube Map, storing 6 square perpendicular rectilinear projections. You still have some distortion where lines of lower resolution warp around the view, but you have the advantage of very efficient display, the ability to easily insert your images/video into existing engines, and the ability to use the huge library of existing cubemaps as test data (without having to go out and capture reference and calibration footage).
I have actually rendered to cubemaps many years ago when I made Half-life maps, with custom skyboxes. Even that felt awesome, to have your own environment surround the level, cannot imagine what real VR will make it feel like.
But yes, cubemapping it all feels like a good way to do it, as you get six squares which fill a rectangular frame just fine. And the baked in distortion is not as extreme as rendering it as at the poles/zenith&nadir when rendering as a panorama. And when rendering it in the player you only have to use a cube with 12 triangles compared to a sphere which to avoid anomalies in the image should have as many triangles as possible. A sphere mode would probably be nice to have though, as I assume many existing 360x180 media is stored like that outside of game content.
I wonder how video-codecs would treat a feed with six squares with sharp contrast at the edges, hehe. To create media like that I imagine a camera that is a cube, with 90+ degree lenses, placed on a thin steel rod so the cube itself looks like it is balanced on one corner. Hmmm. Will make for a brain-mess to try to sync/stitch/align all that, haha. Oh, my head is spinning. Wish I worked in a vision laboratory.
