Page 2 of 3

Re: A simple Virtual Reality media player project

Posted: Tue Apr 30, 2013 9:25 pm
by StephaneLX
Hello t0pquark,
gravity360 wrote:Again: not the "Be all, End all" solution, but given the known restrictions, would this be a step forward?
I think that mixing static and dynamic content is a great idea.
In the current version of VR Player, I decided not to allow loading multiple media files at once because I was afraid of performances issues but combining images with a video should not be a problem. This is some thing I will keep in mind when I will work on the layered video feature.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Tue Apr 30, 2013 9:28 pm
by StephaneLX
Quick update about VR Player status:
The oculus rift head tracking is now working! :D
You must select the following option in the main menu: Tracker->Oculus Rift

-Stephane

Re: A simple Virtual Reality media player project

Posted: Wed May 01, 2013 3:47 am
by robbyrob
StephaneLX wrote:Quick update about VR Player status:
The oculus rift head tracking is now working! :D
You must select the following option in the main menu: Tracker->Oculus Rift

-Stephane
And it works good. Just tested it with some video. I could not find a option, but do you think to implement movement into your player? asking because i would like to change the size of the movie-plane, it was like sitting in the front row in a cinema.

Re: A simple Virtual Reality media player project

Posted: Wed May 01, 2013 7:22 am
by StephaneLX
Hi robbyrob,

You can move around by using the arrows keys and pageup/pagedown buttons.
Also, you can use F1 to calibrate orientation and F2 to reset orientation+position.
Space bar is for play/stop.

The keys are predefined but in future version, I will implement key mapping for customization and the ability to use other devices like the xbox controller.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Wed May 01, 2013 10:05 am
by gravity360
AWESOME! I absolutely love this project! Keep it up!
Soon you could build you're own VR Theater! Wonder what the show times will be? jk

Re: A simple Virtual Reality media player project

Posted: Wed May 01, 2013 10:55 am
by StephaneLX
Thanks gravity360, I appreciate the feedback!
gravity360 wrote:Soon you could build you're own VR Theater!
It would be nice but there is already a forum member who is working on it. You should check out his virtual cinema project.
http://www.mtbs3d.com/phpbb/viewtopic.php?f=140&t=15411

With VR Player, I am trying to focus on the movie immersion. The ultimate goal is to feel like being in the movie rather than being in an environment where you are watching a movie.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 12:49 am
by StephaneLX
Hi,

This week I uploaded a video on youtube to demonstrate the Oculus integration with VR Player and I received very positive feedback.
http://www.youtube.com/watch?v=nRKkstYqHV0

I am very grateful but this made me realized that I included a lot of features in VR Player that are not as intuitive as I thought..
Using videos for presenting use cases is the most efficient way I found yet to generate constructive feedback because users don't have to play with config, find content etc..
So I decided to repeat the experience more often starting with 2 features that I really like:

Support for surround sound
http://www.youtube.com/watch?v=f95u8sC_tCc

Support for stereoscopic movies with wide angle
http://www.youtube.com/watch?v=pd3AMWrjc4I

Thank you again for your feedback,

-Stephane

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 12:56 am
by Skaven252
This is an awesome piece of software. It's looking better and better with every version. It allows me to view any format 3D movie on the Rift in a way that actually works. :D Thanks for your great work!

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 1:14 am
by Skaven252
A couple of thoughts for the backlog (you probably have these already?):
  • Ratio adjustment for all projections. Many side-by-side 3D videos are in half width pixel aspect. Currently only the plane projection's ratio can be adjusted (the ratio for a 16:9 half width side by side video is 1.7777777...)
  • Presets. You usually need to change quite a few settings to get the video working, but then there may be more similar videos. It could be handy if you could save collections of settings as a preset.
  • Tracker reset. I got my Rift's tracker somehow stuck so that I had to look to the right to see the forward view. The only way I found to reset the tracker was to restart VRPlayer. (maybe also make the tracker gamepad / mouse adjustable even in Rift mode?)

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 1:18 am
by Skaven252
Skaven252 wrote:The only way I found to reset the tracker was to restart VRPlayer
Oh... read the documentation. Press F1 to reset. :D

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 3:09 pm
by IsoMacintosh
Tried the experimental surround sound. Well it sounds experimental alright.

http://www.youtube.com/watch?v=0-vCa3HkhhM

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 3:39 pm
by rramirez125
I am a photographer, videographer more than a coder and I am trying to build a camera with 12 gopro 3 that would allow me to record video in 360 3d to be used by the oculus (kind of like this http://freedom360.us but 2 cameras per side instead of one)
I have not been able to find what would the actual output have to be to play the limit size of the image in oculus, if I have the two 3d images 360 x 180 image for a single frame of the video) what would the size had to be? and also the size for each of the squares?, and what program you suggest I use to go from the ready frames to the movies that can be read by the VR player.
thanks for any help you can provide and great program this is exactly what I was looking for.

Re: A simple Virtual Reality media player project

Posted: Sun May 05, 2013 4:26 pm
by geekmaster
rramirez125 wrote:I am a photographer, videographer more than a coder and I am trying to build a camera with 12 gopro 3 that would allow me to record video in 360 3d to be used by the oculus (kind of like this http://freedom360.us but 2 cameras per side instead of one)
I have not been able to find what would the actual output have to be to play the limit size of the image in oculus, if I have the two 3d images 360 x 180 image for a single frame of the video) what would the size had to be? and also the size for each of the squares?, and what program you suggest I use to go from the ready frames to the movies that can be read by the VR player.
thanks for any help you can provide and great program this is exactly what I was looking for.
I am using 12 pixels per degree for a one-to-one pixel mapping between windows desktop and Rift DK virtual environment, in my GMsphere program. That 12 PPD was chosen to make the pixels look like you are viewing them at a great distance with the naked eye. More PPD looks like you are looking through high-magnification binoculars. Less PPD looks like the projection sphere is moving with you, instead of part of the external environment. That virtual "magnification" effect is caused by a small head movement causing a large number of pixels to fly across you FoV, just like when searching for a visual target through binoculars. For me, 12 PPD looked nice, but I only tested integer values.

As a nice coincidence, typical 360-degree panorama images are stored as 4096x2048, which is nearly the same PPD (4096/360 = 11.4) as I determined experimentally (12).

Of course, it is not necessary to actually record at such a high resolution (especially for video), because the resolution can be interpolated up to 4096 while mapping it into the VR environment. Scaling an image up or down does not affect how you perceive it in VR (other than its size). Even 1920-wide will wrap around you such that you need to turn your head to see the sides (unless it is scaled down). 1920-wide is nice for typical 1080p content to minimize visible letterboxing at the top or bottom though, which I determined by watching 1080p SBS 3D "Sintel" in my GMsphere "PixelBall" environment (using modified GMphere code compiled directly into mplayer source code, with head tracking).

So, if you want your video to appear attached to the environment instead of your head, and to not appear to be viewed through binoculars, try using about 12 pixels per degree of angular resolution (scaled to fit with the Rift DK optimal 110-degree FoV).

EDIT: I analyzed the code I borrowed from (Deskope) more thoroughly, and it looks like the PPD value may actually be pixels per TWO degrees, which means the best actual PPD in my experience may be about 6 (close to Michael Abrash's claim of 14 pixels per two degrees).

Re: A simple Virtual Reality media player project

Posted: Mon May 06, 2013 6:26 am
by StephaneLX
Hello Skaven252,
Skaven252 wrote:A couple of thoughts for the backlog (you probably have these already?):
Ratio adjustment for all projections. Many side-by-side 3D videos are in half width pixel aspect. Currently only the plane projection's ratio can be adjusted (the ratio for a 16:9 half width side by side video is 1.7777777...)
Presets. You usually need to change quite a few settings to get the video working, but then there may be more similar videos. It could be handy if you could save collections of settings as a preset.
Thank you for you suggestions! Some items are already in my back log but I like when people ask for features because it helps me to adjust priorities.
About the ratio of projections, I will add the cylinder height in the next release and maybe dome depth a bit later. I will not modify sphere and cube projections because I did not see medias supporting different ratios (Rectangular Cuboid and Ellipsoid)..
For the settings, currently, it's possible to save some settings in the xmp metadata of files but not the parameters in plugins panels. This issue is my top priority but I like the preset idea and I will look into it right after.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Mon May 06, 2013 6:34 am
by StephaneLX
Hey IsoMacintosh,
IsoMacintosh wrote:Tried the experimental surround sound. Well it sounds experimental alright.
Your video really made me laugh :lol: The example is too perfect.

The surround sound feature is indeed experimental because I did not test with a lot of audio codecs.
I mainly tested with AC3 and 2, 4 or 6 channels.

The result you are experiencing is clearly a bitrate problem but it would help me if you can provide more details on the audio codec you are using + settings.

Thank you,

-Stephane

Re: A simple Virtual Reality media player project

Posted: Mon May 06, 2013 9:31 am
by IsoMacintosh
StephaneLX wrote: The result you are experiencing is clearly a bitrate problem but it would help me if you can provide more details on the audio codec you are using + settings.
after testing i have found the problem to be aac sound. here's some testing results with different files.
  • dolby digital = works perfectly
    dts = moving view has no effect
    aac audio = distorted audio and moving view has no effect
    PCM mono = moving view has no effect
also i think you should add binaural features. maybe use some open source sound engine like this
Another way is to use dolby headphone and do a wrapper like this
or just tell people to buy a soundcard with dolby headphone support. :D

Re: A simple Virtual Reality media player project

Posted: Mon May 06, 2013 9:49 pm
by StephaneLX
Hello rramirez125!
rramirez125 wrote:I am a photographer, videographer more than a coder and I am trying to build a camera with 12 gopro 3 that would allow me to record video in 360 3d to be used by the oculus (kind of like this http://freedom360.us but 2 cameras per side instead of one)
This is a very interesting project but quite a challenge! (I wish I had 12 gopros to make some tests :lol:)
rramirez125 wrote: I have not been able to find what would the actual output have to be to play the limit size of the image in oculus, if I have the two 3d images 360 x 180 image for a single frame of the video) what would the size had to be?
If you want to use a spherical 360x180 video and would like high resolution, GeekMaster suggestion of 4096x2048 is an excellent choice.
For stereo format, you have to combine both of your videos (left and right eye) in over/under format.
This will create a 4096x4096 video! It works but this creates huge files and finding a good codec for compression at this resolution is hard..
I would appreciate to have the advice/suggestions of a codec expert :)
For the current generation of Rift it's probably over kill but keep in mind that the consumer version will probably have higher resolution.

Note: If possible, keep resolution size as a power of 2 because at higher resolution, it's been reported that you might experience memory issues.
rramirez125 wrote: what program you suggest I use to go from the ready frames to the movies that can be read by the VR player.
I know there are a couple of applications available but I prefer not to recommend a specific one because I don't have a lot of experiences with stitching software. I am curious and if you find a good one, you can share the information on this board!

Good luck!

-Stephane

Re: A simple Virtual Reality media player project

Posted: Tue May 07, 2013 3:38 pm
by polygonwindow
This software was recently released.

http://www.video-stitch.com/

Haven't tried it yet, but if you have an nvidia gpu it can accelerate the stitching.

Re: A simple Virtual Reality media player project

Posted: Tue May 07, 2013 6:06 pm
by geekmaster
StephaneLX wrote:For stereo format, you have to combine both of your videos (left and right eye) in over/under format.
This will create a 4096x4096 video! It works but this creates huge files and finding a good codec for compression at this resolution is hard..
I would appreciate to have the advice/suggestions of a codec expert :)
-Stephane
For "normal" HD the codec of choice is h.264 (or AVC), but for 4K video, it looks like h.265 (or HEVC) is becoming a standard codec:
http://www.brightsideofnews.com/news/20 ... 0mbps.aspx

The h.265 codec is specified as using 10Mbps bandwidth (for 4K video), but you can compress tighter than that for lower bandwidth consumption and reduced storage space, at a reduction in quality that may be worthwhile for some applications.

http://www.youtube.com/watch?v=uKY9-nphpHQ
http://www.youtube.com/watch?v=PZP75PEQo6E

Also, there is rarely a need for your video content to exceed the 24FPS framerate used in movie theaters, as long as your head-tracked view of the projection surface is updated at 60Hz (or higher) framerate. This was discussed in great detail in my "PTZ Tweening" thread, and was used in my "pixelpi" program (a subset of GMsphere) released in my "Rift Pi" thread.

Re: A simple Virtual Reality media player project

Posted: Tue May 07, 2013 7:42 pm
by StephaneLX
@IsoMacintosh & GeekMaster

Thank you both for the infos on audio and video codecs. This is helping me a lot.

@polygonwindow

The stitching software got a nice list of features, I will check this out. Thank you.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 7:59 am
by polygonwindow
This is some great work your doing Stephane! Some feedback and ideas:

Soon, Ill be starting to shoot content for your player. I'm super excited about the possibilities. Ill be using this rig: http://freedom360.us/ which was mentioned in the previous posts.

here is an example video from the rig:

http://we.tl/4Jm6LIFpjM

It can shoot at 5.7k, but I think a more ideal format for content will be 4k. Stitched, the video is 4000x2000. Looks incredible on your player with the rift! The video itself plays back fine on my system (i5 4.2ghz, ati 6870), though the rift tracking is a bit laggy. Do you think this is due to the high resolution of the video? Also, when i size the window to be almost full screen, there are no playback issues, but when i hit full screen the frame rate drops by about half.

Although Ill be working with a sphere of video, is there any way to set the player to load the video so that it faces in a direction I want it to for the viewer? ie, a video of a car chase, and I want the video to load so that the 'looking forward' view is oriented out of the passenger window...or in that example video, lets say I want the viewer to be 'looking forward' at the hang glider guy and not the horizon. Ideally I want to set this for the viewer this will be highly useful in guiding the viewer to look 'more or less' at what I want them to look at.

Perhaps instead of having lots of options for which unwrapper, sphere slices, etc..we can just base that around which camera the footage was shot with and imbed that info in the video file itself or in an accompanying txt so that its always playing back the 'best' rather than having to tweak for each clip? Will this be workable via your plugins perhaps?

Finally, for future versions, it would be really sweet to be able to control playback with the rift itself. The Proton Pulse demo has a very simple implementation of selecting stuff, but its highly intuitive and works well. In a perfect world, I want to be able to select videos, pause, stop, play etc without having to take the rift off. I was thinking, if you tilt your head and look at the ground for a second or so, a menu could fade in and give you those options? Users would likely never be looking at the ground for long anyways. or perhaps head gestures like nodding or shaking your head to move forward or back in the menus? I realize this may be a challenge.

Ill be donating to your project soon...this is great stuff!!

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 11:12 am
by geekmaster
polygonwindow wrote:It can shoot at 5.7k, but I think a more ideal format for content will be 4k. Stitched, the video is 4000x2000.
Hmm... The 360-stereoscopic image sets I have been working with are 4096x2048 (or other power-of-two sizes), which makes processing them a lot simpler. Using decimal-based sizes is not efficient for a binary-based computer (especially when working with lower-power hardware like the Raspberry Pi or Ouya, both of which I plan to support).

Although besides the still image panorama "standard" that uses power-of-two dimensions, there are also the "4K Video STANDARDS" to deal with:
Full Aperture 4k: 4096 x 3112 (12,746,752 pixels)
Academy 4k: 3656 x 2664 (9,739,584 pixels)
Digital Cinema 4k: 4096 x 1714 (7,020,544 pixels)
Digital Cinema Aperture 4k: 3996 x 2160 (8,631,360 pixels)
Your 4000x2000 resolution does not match any of the common standards for 360-images or for 4K video.

Are you sure about the missing 96 rows and 48 columns of pixels in your "4K" stitched video? IMHO, it would be MUCH better to use one of the existing 4K standards rather than creating a new NONSTANDARD (decimal-based) 4K resolution. Few things in the computer world use resolution dimensions based on the human decimal number system.

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 11:47 am
by StephaneLX
Hi polygonwindow!
polygonwindow wrote: Looks incredible on your player with the rift! The video itself plays back fine on my system (i5 4.2ghz, ati 6870), though the rift tracking is a bit laggy. Do you think this is due to the high resolution of the video?
The resolution may slowdown the system and impact the tracking but the real issue is in the implementation.
Technically, I am using a timer to poll tracking data each x milliseconds but in the current implementation, the polling has a low priority. This imply that the player tend to skip polling too often, especially when the system is busy. I will release a quick fix soon.
polygonwindow wrote: Also, when i size the window to be almost full screen, there are no playback issues, but when i hit full screen the frame rate drops by about half.
What is the resolution of your screen? Did you change projection settings (Ex: Increased the number of slices/stacks of a sphere)?
GeekMaster is correct about the resolution of the video, this may cause problems.
polygonwindow wrote:Perhaps instead of having lots of options for which unwrapper, sphere slices, etc..we can just base that around which camera the footage was shot with and imbed that info in the video file itself or in an accompanying txt so that its always playing back the 'best' rather than having to tweak for each clip? Will this be workable via your plugins perhaps?
You can add the information in the xmp metadata of the media file. I created a custom Adobe panel to simplify the process. (See VRPlayerXmpTools as attachment)
I will try to produce a quick you tube video tutorial about this functionality.
StephaneLX wrote:Although Ill be working with a sphere of video, is there any way to set the player to load the video so that it faces in a direction I want it to for the viewer? ie, a video of a car chase, and I want the video to load so that the 'looking forward' view is oriented out of the passenger window...or in that example video, lets say I want the viewer to be 'looking forward' at the hang glider guy and not the horizon. Ideally I want to set this for the viewer this will be highly useful in guiding the viewer to look 'more or less' at what I want them to look at.
Thank you for the suggestion, this will be added in the metadata!
StephaneLX wrote:I want to be able to select videos, pause, stop, play etc without having to take the rift off. I was thinking, if you tilt your head and look at the ground for a second or so, a menu could fade in and give you those options?
I am currently refactoring the application's code to stabilize and improve the performances but UI improvement should be one of the next logical step! This is some thing I like to do and there is so many thing I want to experiment!

Thank you very much for your support!!
If you have other questions/comments, do not hesitate.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 12:36 pm
by polygonwindow
geekmaster wrote:
polygonwindow wrote:It can shoot at 5.7k, but I think a more ideal format for content will be 4k. Stitched, the video is 4000x2000.
Hmm... The 360-stereoscopic image sets I have been working with are 4096x2048 (or other power-of-two sizes), which makes processing them a lot simpler. Using decimal-based sizes is not efficient for a binary-based computer (especially when working with lower-power hardware like the Raspberry Pi or Ouya, both of which I plan to support).

Although besides the still image panorama "standard" that uses power-of-two dimensions, there are also the "4K Video STANDARDS" to deal with:
Full Aperture 4k: 4096 x 3112 (12,746,752 pixels)
Academy 4k: 3656 x 2664 (9,739,584 pixels)
Digital Cinema 4k: 4096 x 1714 (7,020,544 pixels)
Digital Cinema Aperture 4k: 3996 x 2160 (8,631,360 pixels)
Your 4000x2000 resolution does not match any of the common standards for 360-images or for 4K video.

Are you sure about the missing 96 rows and 48 columns of pixels in your "4K" stitched video? IMHO, it would be MUCH better to use one of the existing 4K standards rather than creating a new NONSTANDARD (decimal-based) 4K resolution. Few things in the computer world use resolution dimensions based on the human decimal number system.
Actually, I was incorrect...when I said 4k, I suppose I meant 'somewhere around 4k' hahah. The resolution of my example shot is 4048x2024. Does that make a difference versus the resolution you've been working at?

@Stephane
Great reply. I'll keep playing with it and fire over any other ideas that come to mind!

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 2:03 pm
by geekmaster
polygonwindow wrote:Actually, I was incorrect...when I said 4k, I suppose I meant 'somewhere around 4k' hahah. The resolution of my example shot is 4048x2024. Does that make a difference versus the resolution you've been working at?
It does not really make a difference, because I need to sample my input images at whatever input resolution I have available. I just thought that we do not need yet another "standard" 4K resolution. :D
The nice thing about standards is that there are so many of them to choose from.
- Andrew S. Tanenbaum
Image
http://xkcd.com/927/

Re: A simple Virtual Reality media player project

Posted: Thu May 09, 2013 8:20 pm
by StephaneLX
StephaneLX wrote:You can add the information in the xmp metadata of the media file. I created a custom Adobe panel to simplify the process. (See VRPlayerXmpTools as attachment)
I will try to produce a quick you tube video tutorial about this functionality.
Here's the video:

http://www.youtube.com/watch?v=AWbCKWW3Lgg

This video contains 3 examples:
-A dome projection (side by side)
-A Plane projection (over/under)
-A sphere projection (Mono)

More settings will be added in next versions. :)

-Stephane

Re: A simple Virtual Reality media player project

Posted: Fri May 10, 2013 7:44 am
by polygonwindow
Thanks Stephane I will give this a try!

Re: A simple Virtual Reality media player project

Posted: Fri May 10, 2013 8:58 am
by rramirez125
Thank you guys for your reply i will let you know how it goes with building my 360 3D camera.

Re: A simple Virtual Reality media player project

Posted: Sat May 11, 2013 3:38 pm
by rramirez125
I am working on the 3d Cad design of the rig for the 12 cameras setup for the 360 video in 3d,
I read in a previous post in this thread that the distance between the centre of the two lenses was suppose to be 70 mm,
then somewhere else that oculus rift distance between lenses was 64 mm.
does anybody know what the optimal distance between the centre of the lenses should be?
thanks in advance
Rafa

Re: A simple Virtual Reality media player project

Posted: Sat May 11, 2013 3:55 pm
by geekmaster
rramirez125 wrote:I am working on the 3d Cad design of the rig for the 12 cameras setup for the 360 video in 3d,
I read in a previous post in this thread that the distance between the centre of the two lenses was suppose to be 70 mm,
then somewhere else that oculus rift distance between lenses was 64 mm.
does anybody know what the optimal distance between the centre of the lenses should be?
thanks in advance
Rafa
When both cameras are pointing the same direction, you use the IPD of the lens center. But when fisheye camera viewing angles diverge you need to use a portion of your image pair from a pair of the cameras, where the left image comes from the right half of the left camera, and the right image comes from the left half of the right camera. So in this case, the lens centers will be farther apart than the viewer IPD. Perhaps 70mm lens center distance is correct for a 64mm viewer IPD, but it depends on the lenses and camera settings as well.

There are a number of good documents on the web that describe this, such as this:
Capturing Omni-Directional Stereoscopic Spherical Projections with a Single Camera
http://www.ivec.org/sites/default/files/vsmmpaper.pdf
(see figure 5 in that PDF file)

For stereoscopic panoramas from fisheye lenses, many more photos are recommended than for non-stereoscopic panoramas:
http://www.mtbs3d.com/phpbb/viewtopic.php?f=26&t=11039

Re: A simple Virtual Reality media player project

Posted: Sun May 12, 2013 4:09 pm
by rramirez125
Thank you Geekmaster I appreciate all the useful information you provide

Re: A simple Virtual Reality media player project

Posted: Sun Jun 09, 2013 1:49 am
by StephaneLX
Hi!

I just uploaded the latest version of VR Player (0.3) on http://vrplayer.codeplex.com

In this release I took the time to refactor the plugins system (something I wanted to do for a long time).
For most users, there is no impact on the software beside the folder structure.
For developer, this will gives the tools to integrate custom tracking devices, try new type of projections matching specific capture devices and add special effects.
For non-developer, I created a new projection plugin that allow file import from Maya and 3ds (Including geometry and UV mapping).

Since the last release, I have experimented with a lot with positional tracking.
I added a TrackIR plugin and I received help to correct Razer Hydra jitter problems by applying a smoothing filter.
Positional tracking add an interesting dimension and feel more natural when watching a video on a virtual screen but for spherical/dome videos, this cause deformation so in these cases, it's better not to move the head too much.

Tracking performances are a little better but there is still plenty of room for improvement. This is something I take seriously and I will continue invest time and efforts.

Currently, my backlog is getting bigger and bigger. I received a lot of great ideas and I appreciate it!
I'm doing my best to keep up :)

Note that this is still an alpha version so if you find bugs or any issues, please contact me

Finally, I hope to see more immersive content soon!
Not only sport and music events but also short films including storytelling..
It's true that stereoscopic spherical movies are complicated to produce but there are good alternatives available now.
Stereoscopic video projected on a dome are very immersive and accessible with cameras like dual go pros.
Normal spherical videos are also a lot of fun to watch with the Rift.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Sun Jun 09, 2013 11:57 am
by Skaven252
StephaneLX wrote:Since the last release, I have experimented with a lot with positional tracking.
I added a TrackIR plugin and I received help to correct Razer Hydra jitter problems by applying a smoothing filter.
On a related topic; I got this idea and I was wondering... many viewers have complained about shaking and head movement in my POV videos.

VirtualDub has this plugin called Deshaker, which can analyze shake and motion in video, and then compensate for it. The user first runs a video analysis pass, which produces a vector data file (motion vectors per frame). The user can then choose to re-encode the video with different anti-shaking options. But this usually applies a quite heavy crop to the video.

Would it be a far fetched idea to read this vector file into VRPlayer, and make it compensate for the shaking in real time? You could see the virtual screen waggle around, but the picture would be more still. The viewer could follow the screen at their own pace, rather than being constantly shaken around by the camera man's whims.

Could this work? Or would it take the viewer just closer to the Uncanny Valley of Nausea? :)

The Voodoo Camera Tracker could be another interesting option - it would do it in 3D. But this means the viewers would have to run after the virtual screen.

Re: A simple Virtual Reality media player project

Posted: Sun Jun 09, 2013 3:08 pm
by BOLL
Skaven252 wrote:Would it be a far fetched idea to read this vector file into VRPlayer, and make it compensate for the shaking in real time?
I actually mentioned this to StephaneLX in a PM xD Though I was mostly daydreaming hehe.

My new daydreaming is to connect an IMU to an Arduino and output the tracking data as audio to one of the cameras. I have no idea if it's even possible, but it doesn't sound... too far fetched, does it? Haha :x It would still mean the player would need support for it, perhaps there is some standard for stuff like this... I will try to research this when I get time, it popped into my head earlier today :)

Re: A simple Virtual Reality media player project

Posted: Sun Jun 09, 2013 9:48 pm
by StephaneLX
Hi Skaven252!

I had the discussion with Boll and this was my answer:
This is some thing I wanted to try because one of the firsts experiments I made with the player was on a stationary bike, watching a mountain biking video and it was very shaky. I talk about the idea with video professionals and they said it was possible to stabilize the video source instead and get acceptable results so I did not invested more time on the feature. It's still in my back log and it would be easy to move the camera based on a stream of data synced with the video. The complicated part is to find/create a standard way to do it. If go pro cameras had an accelerometer and was embedding data in metadata or a sidecar file, it would be perfect. Anyhow, the feature is on hold util I receive more demands.
I was thinking about processing raw data from a sensor in a camera for stabilizing position and also orientation but I did not thought about using the output files from a stabilization software. I really like the idea.

If you have a sample movie the associated data file including a list of vectors with maybe time stamps and optionally quaternions for orientation, I can try to implement the camera movement during video playback.

-Stephane

Re: A simple Virtual Reality media player project

Posted: Sun Jun 09, 2013 10:12 pm
by geekmaster
Skaven252 wrote:VirtualDub has this plugin called Deshaker, which can analyze shake and motion in video, and then compensate for it. The user first runs a video analysis pass, which produces a vector data file (motion vectors per frame). The user can then choose to re-encode the video with different anti-shaking options. But this usually applies a quite heavy crop to the video.

Would it be a far fetched idea to read this vector file into VRPlayer, and make it compensate for the shaking in real time? You could see the virtual screen waggle around, but the picture would be more still. The viewer could follow the screen at their own pace, rather than being constantly shaken around by the camera man's whims.

Could this work? Or would it take the viewer just closer to the Uncanny Valley of Nausea? :)

The Voodoo Camera Tracker could be another interesting option - it would do it in 3D. But this means the viewers would have to run after the virtual screen.
I have always used the Deshaker plugin in VDub to create a larger image, filling in the edges with content from past and future frames. Even when I had that off, I still used a bigger output frame size, and my input rectangle just rotated and moved around inside that. It would only crop in extreme cases where the input frame momentarily jumped partly off the output frame.

Now, regarding reusing the motion vectors from Deshaker, I absolutely LOVE that idea! I am very impressed! I really enjoy discovering simple ideas that I have not already considered myself at some point in the past. Thank you VERY much for sharing this new (to me) idea! It is the norm to find "new" ideas presented here that I read about long ago (and sometimes can find again with Google). But this one really is new to me, and I *LOVE* learning new things whenever I can.
:D

As I have mentioned in previous posts, MPEG files also contain embedded motion vectors, and these may also be useful for a similar purpose, without needing to use a separate Deshaker pass (but only in combination with a modified MPEG codec to provide such motion information). But using Deshaker data is much more immediately useful, so I will add that to my extensive "To Do" list...
:o

Re: A simple Virtual Reality media player project

Posted: Mon Jun 10, 2013 5:59 am
by StephaneLX
Hey Boll,

I am not sure what you mean by:
Skaven252 wrote:My new daydreaming is to connect an IMU to an Arduino and output the tracking data as audio to one of the cameras.
Do you want to generate audio similar to what a Theremin produce? What will be the visual output in the camera?

-Stephane

Re: A simple Virtual Reality media player project

Posted: Mon Jun 10, 2013 6:45 am
by BOLL
StephaneLX wrote:Do you want to generate audio similar to what a Theremin produce? What will be the visual output in the camera?
The thought would be to save stabilization data from the IMU as an audio-signal in one of the video-feeds in a stereo camera setup where both cameras can record stereo audio. I realized I'm not a sound engineer and have no idea if it is possible to extract enough different frequencies from two channels to actually store enough data for it to make sense :P And compression in the camera might ruin it anyway. The idea probably comes from various iPhone accessories which communicates through the audio port. It would automatically be synced with the video though, which would be nice.

I'd say no need to bother with that, my first idea was to use a deshaker, just like we've talked about, what made me hesitant was that I have no idea of how it handles stereoscopic video. Would it work just running deshake for one video at the time? As the two cameras are fixed, no convergence for Rift video, it would seem to be a fairly straight forward thing, but the two views will get quite different stabilization in certain situations. Imagine that the camera rig has been rotated around one of the cameras, the stabilization for the two would be quite different...

With the cameras on fixed on a baseline at least one of the cameras will always have translation which would have to be taken into consideration when moving the views in the player :P No? My brain is melting now and I need to get back to work, heh!

Re: A simple Virtual Reality media player project

Posted: Mon Jun 10, 2013 1:16 pm
by Skaven252
All right! I'll whip up a video file. I don't know if Deshaker can properly analyze side-by-side video, but I'm hoping that single video (left only) will be enough. also, I don't know how Deshaker behaves with text (especially if it's scrolling), so I'll make the test data with a single take/cut from my "Norrtull flea market" video.

StephaneLX, if you send me your email address in a private message, I'll deliver the files to you when I'm done. I'll send the side by side 3D video (with full soundtrack), the left only video (with rough action cam audio) and the Deshaker log file. The Deshaker data is in text format, so it should be parseable.

Does anyone know if there's a way to crop and stretch an already rendered video in VirtualDub so that I could feed just one side of it to Deshaker? (I couldn't find a way to do it at a glance)

I'm not sure how to deal with the fisheye distortion. The action camera footage is fisheye warped, which matches fairly well with the Rift - but since the virtual screen is now going to move around to compensate for shaking, maybe the video should be de-fished (to rectilinear) and the fisheye warp applied in post?

Re: A simple Virtual Reality media player project

Posted: Wed Jun 19, 2013 7:19 am
by geekmaster
Skaven252 wrote:... I'm not sure how to deal with the fisheye distortion. The action camera footage is fisheye warped, which matches fairly well with the Rift - but since the virtual screen is now going to move around to compensate for shaking, maybe the video should be de-fished (to rectilinear) and the fisheye warp applied in post?
I believe VDub has crop and stretch filters built in. And yes, you need to de-fisheye to deshake unwanted camera motion, then re-fisheye the result. To use the fisheye filters, you need to split the video into left and right videos, defisheye both of them, join them SBS, deshake them, split them again, fisheye them, and rejoin them. Deshaking them independently will cause arbitrary vertical misalignment at times, so they need to be done joined SBS. Defisheye and fisheye use the same filter, just with different coefficients (i.e. adjust slider between degree of barrel or pinchushion required). I used deshaker on a lot of my videos about 7 or 8 years ago for video filmed in moving vehicles. But I prefer the deshaker mode that fills in the edges with pixels from past and future frames, instead of a moving black border frame.

I often used VDub filters in AVISynth for larger projects with more operations, because it is scriptable and allows you to batch a lot more stuff together, and provides additional features not in VDub. For the steps described above, I would probably use AVISynth with Deshaker, instead of VDub.

I also used another great tool back then that inserted motion-vector based interpolation frames (where the key frames were supplied by the original video). It did a wonderful job of creating a higher framerate, or of converting a video to extreme slow-motion (up to 10x) with no visible artifacts. I cannot remember what it was called, nor can I find it again, but it was a wonderful tool (free for non-commercial use).

Some quick Google results to answer your questions:

VDub stretch:
http://www.kevblog.co.uk/virtualdub-res ... r-youtube/

VDub crop & resize:
http://www.youtube.com/watch?v=NF2m7DDw9KY

Hint: You can crop (including cropping a single SBS half) with the Cropping button in filter select. If not using other filters, select the Null Transform filter.