A simple Virtual Reality media player project

Skaven252
Cross Eyed!
Posts: 120
Joined: Thu Aug 02, 2012 2:07 am
Location: Sweden

Re: A simple Virtual Reality media player project

Post by Skaven252 »

Does Deshake work better with rectilinear (non fisheye) footage? I reckon it might, as the warping may confuse the motion detectors. However you can limit Deshaker to the middle of the screen to ignore the most warped parts of the image.

I could do the deshake non-destructively by running Deshaker to an AviSynth feed that has the defisher in it, just to produce the .log file but apply it to the original video.

VRPlayer actually allows to match the FOVs of different action cameras by adjusting the FOV of the viewer. I've managed to get fisheyed footage look more or less rectilinear on on the Rift with the right settings.

I'd prefer not to de-fish let alone re-fish, because defishing loses information from the corners of the image, and I reckon re-fishing would lose even more. By not de-fishing the video there's more left to see when looking around.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: A simple Virtual Reality media player project

Post by geekmaster »

Without defisihing you would get the effect of a bouncing fisheye lens between the eye and the scene. The undesirable warp would be most severe near the edges during a bounce, but could invade the safe area during large bounces (when deshaking causes perceived blur on high-contrast edges in the direction of the bounce).

If you keep physical shaking to a minimum, it may work well enough to deshake the fisheye content directly.

For more severe shaking, you could get fancy and do a pipelined single-pass per-pixel combined defish-deshake-refish pass where each pixel only gets moved once. Perhaps compute a per-frame displacement map from the deshaker data and (before and after) fisheye lens radial distorion formula?
Skaven252
Cross Eyed!
Posts: 120
Joined: Thu Aug 02, 2012 2:07 am
Location: Sweden

Re: A simple Virtual Reality media player project

Post by Skaven252 »

How bad is it, really, if I leave the cameras' own fisheye into the video, to retain as much of the image as possible?

Would you be willing to verify it yourself? You can download a 60fps 170 degree FOV video from here (1.1 GB) and try viewing it with VRPlayer. Use Planar projection with ratio 1.77777. Disable VRPlayer's own barrel distortion, and tweak the FOV until the curved lines on the sides look more or less straight on the Rift.

It seemed at least to me that just by tweaking the FOV of VRPlayer you can find a suitable match for different action cam footage that all have different FOVs. Assuming that all the involved lenses (on the Rift and in the action cameras) are spherical.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: A simple Virtual Reality media player project

Post by geekmaster »

Skaven252 wrote:How bad is it, really, if I leave the cameras' own fisheye into the video, to retain as much of the image as possible?

Would you be willing to verify it yourself? You can download a 60fps 170 degree FOV video from here (1.1 GB) and try viewing it with VRPlayer. Use Planar projection with ratio 1.77777. Disable VRPlayer's own barrel distortion, and tweak the FOV until the curved lines on the sides look more or less straight on the Rift.

It seemed at least to me that just by tweaking the FOV of VRPlayer you can find a suitable match for different action cam footage that all have different FOVs. Assuming that all the involved lenses (on the Rift and in the action cameras) are spherical.
I just tried your recommended settings. It looks like a Movie screen that does not completely fill my FoV (A-cups, eyelashes brushing lenses), with a little pincushion distortion. To minimize pincushion, you need a little more fisheye, perhaps by zooming out a bit on your cameras if possible. Otherwise VRPlayer needs to apply a small amount of barrel distortion.

Watching this in my Rift with spherical projection, barrel distortion 1.5 looks pretty cool (except when looking straight down or up). I can look around just like being there that way, instead of watching it on a virtual movie screen. When you move the cameras, that is a little "dizzy-making" though, but slow camera movements are much more tolerable than fast. And of course, that is much less of a problem in less-immersive "movie screen" mode...
Skaven252
Cross Eyed!
Posts: 120
Joined: Thu Aug 02, 2012 2:07 am
Location: Sweden

Re: A simple Virtual Reality media player project

Post by Skaven252 »

Thanks for testing it and giving feedback, geekmaster!

An upcoming version of VRPlayer will have new projection modes: Dome and Cylinder, with angle/FOV cropping. These may give some additional options to get the distortion just right.

The pixel shader barrel distortion unfortunately blurs the video a bit too much for my liking, that's why I'm hoping it could be achieved with geometry instead. The blurring depends on the scale and distance of the screens; at some point the MIP-map kicks in and the video/texture loses half of its resolution.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: A simple Virtual Reality media player project

Post by geekmaster »

Skaven252 wrote:... The blurring depends on the scale and distance of the screens; at some point the MIP-map kicks in and the video/texture loses half of its resolution.
I actually prefer a blend between MIP mapping and Monte Carlo (random) sampling, which gives some temporal dithering without blocky edges or too much MIP map smoothing. The problem with MIP maps is incorrect blurring for oblique-angle views. RIP maps are better:
http://en.wikipedia.org/wiki/Mipmap

The problem is that some of the cool things I have been doing with CPU and framebuffer may not be easy to implement using shaders. But I plan to try anyway, when I get around to it...
User avatar
goodl
One Eyed Hopeful
Posts: 19
Joined: Mon Jul 08, 2013 7:51 am

Re: A simple Virtual Reality media player project

Post by goodl »

I have donated in the hope you will continue development, even in its rough alpha state its probably my most used rift app
StephaneLX
One Eyed Hopeful
Posts: 39
Joined: Mon Sep 03, 2012 10:38 am
Location: Montreal

Re: A simple Virtual Reality media player project

Post by StephaneLX »

Hi goodl!
goodl wrote:I have donated in the hope you will continue development, even in its rough alpha state its probably my most used rift app
Thank you very much for your support, it means a lot to me!
You can rest assured, I am working nights and weekends on this project and I am having a great time while doing it.
I am determined to bring fully immersive videos to the community and it will be crazy with the rift HD. :woot

-Stephane
polygonwindow
One Eyed Hopeful
Posts: 13
Joined: Sat Apr 06, 2013 9:07 pm

Re: A simple Virtual Reality media player project

Post by polygonwindow »

Hey Stephane,

Any chance the next version will have an integrated UI? It would be great to be able to select and control the player using the Rift itself.
StephaneLX
One Eyed Hopeful
Posts: 39
Joined: Mon Sep 03, 2012 10:38 am
Location: Montreal

Re: A simple Virtual Reality media player project

Post by StephaneLX »

Hi!

It took more time than expected but VR Player version 0.4 is online at http://vrplayer.codeplex.com! :D

My favorite features are
- VLC support including live streaming from youtube
- Action camera support with the dome projection

There are some experimental features like stabilizers and process capture that will be improved in future releases.

Here's the complete change log:

-New plugins types in backend: Media and stabilizers
-VLC support (Files, streams/youtube, DVD)
-GDI screen capture
-Dome projection horizontal and vertical crop parameters
-Cylinder projection horizontal crop parameter
-Sphere projection ellipsoid parameters (width, height, depth)
-User settings auto-save
-Keyboard shortcuts edition
-Anaglyph effect
-Cross-eyed layout
-Improved UI
-New sample
polygonwindow wrote:Any chance the next version will have an integrated UI?
Unfortunately integrated UI will not be for this release. I had to make changes under the hood that will facilitate development.

-Stephane
Leahy
Cross Eyed!
Posts: 181
Joined: Thu Jun 07, 2012 1:36 pm
Location: France

Re: A simple Virtual Reality media player project

Post by Leahy »

Really nice work!!! This program seems perfect now for the 3d clips I've been waiting to watch
User avatar
goodl
One Eyed Hopeful
Posts: 19
Joined: Mon Jul 08, 2013 7:51 am

Re: A simple Virtual Reality media player project

Post by goodl »

Loving the new features of the latest release, vlc and stream integration are fantastic
StephaneLX
One Eyed Hopeful
Posts: 39
Joined: Mon Sep 03, 2012 10:38 am
Location: Montreal

Re: A simple Virtual Reality media player project

Post by StephaneLX »

Thank you goodl and leahy! I am glad you like it :D

-Stephane
User avatar
goodl
One Eyed Hopeful
Posts: 19
Joined: Mon Jul 08, 2013 7:51 am

Re: A simple Virtual Reality media player project

Post by goodl »

so you have a projected changelog for the upcoming release Stephane?

Also do you think it would be possible to support the 3d format from a Sony HDR TD10 video camera at some point, I can provide samples?

The format in which the Sony HDR-TD10 and 20 capture 1080p 3D footage, the MVC format, is basically a version of the AVCHD format used by most consumer camcorders although since the HDR-TD10 is a 3D camcorder it captures two streams of 1080p with one stream being the full 1080p image and the other being the difference between the left and right eye.
StephaneLX
One Eyed Hopeful
Posts: 39
Joined: Mon Sep 03, 2012 10:38 am
Location: Montreal

Re: A simple Virtual Reality media player project

Post by StephaneLX »

Hi goodl,
goodl wrote:so you have a projected changelog for the upcoming release Stephane?
I have a backlog full of features and improvements I would like to implement but usually, I don't really keep a change log for the next release.
Right now, I am working on the presets feature because it's a real pain having to change every settings one by one to watch a video clip or a panoramic image. It's also complicated to share the instructions for the users so I hope this will help.
Presets will be saved in a text based format in a file or embeded in xmp metadata.

About the Sony HDR TD10, I am not familiar with the MVC format so I will have to do some reading for the implementation but this is now in my backlog.

Do you know if a Directshow filter that convert a MVC signal to a side by side video exists? This would be nice because VR Player already support directshow and the sbs format.

-Stephane
User avatar
BOLL
Binocular Vision CONFIRMED!
Posts: 295
Joined: Mon Aug 06, 2012 9:26 pm
Location: Sweden
Contact:

Re: A simple Virtual Reality media player project

Post by BOLL »

I finally tried the VR player a while ago Stephane, very nice!

The text format companion file sounds very nice to me :D Just what I had in mind, will make it very easy to tailor content to be viewed as intended! Kind of like a subtitle file :)

Looking forward to experimenting with it later!

Edit: Of course uh... to generate these text files it would be super neat to have some in-app tweaking mode where things like distortion and stuff can be set. Might even be sensible to be able to adjust left/right views separately in case the cameras were misaligned while recording... but that could be a later feature hehe.
StephaneLX
One Eyed Hopeful
Posts: 39
Joined: Mon Sep 03, 2012 10:38 am
Location: Montreal

Re: A simple Virtual Reality media player project

Post by StephaneLX »

Hi,

It will take a bit longer until the next release so until then, here's a new projection plugin for version 0.4.1: the Full Dome Projection (see attachment) This projection is for viewing IMAX and planetarium style videos. Lot of them on youtube!

@BOLL,
BOLL wrote:to generate these text files it would be super neat to have some in-app tweaking mode where things like distortion and stuff can be set. Might even be sensible to be able to adjust left/right views separately in case the cameras were misaligned while recording
What I have until now is a json file with all parameters in the applications.
I created a menu for saving a subset of this information:
-Preset for the current media (projection, effects, stereo mode, stabilizer)
-Preset for the current hmd device (distortion, tracker, layout, fov, horizontal/vertical offsets)
-Presets for the application (keyboard shortcuts, default paths, ...)
-Preset for everything above (backup)
Maybe I will add an other option for saving a camera (fov, position, orientation)

The biggest challenge I am facing right now is trying to make this simple for the content creator. I guess that most of the time people will share media presets only. Once the preset is created, it's super easy viewing a video because everything is setup automatically from the side car file. It will also be possible to embed json data directly in xmp metadata.

An other challenge I have is handling presets for an online stream. I implemented a playlist based on m3u format and when loading a local file, the sidecar file or xmp metadata is read. When loading an url, no metadata so you have to manually open a preset file..
Anyway, I will try to find a solution. Ideas are welcome. :)

-Stephane
You do not have the required permissions to view the files attached to this post.
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: A simple Virtual Reality media player project

Post by Fredz »

Here are lists of fulldome videos I made some time ago (some overlap between the two), nice that you got that implemented :
- http://vrwiki.wikispaces.com/Fulldome+videos
- http://vr.wikinet.org/wiki/Wide_FOV_vid ... 0_fulldome
Nukemarine
One Eyed Hopeful
Posts: 3
Joined: Sun Jul 21, 2013 8:13 am

Re: A simple Virtual Reality media player project

Post by Nukemarine »

As I've mentioned in the other forums, the cylinder projection update has been great for 2d and 3d movie projection and I now prefer it over the flat plane projection. Since its surrounding me, I also notice less impact when drift occurs, though that'll be a non-issue once you implement magnetic correction (probably further down on your to do list). At the moment, I use 0.5 height and 90 degree coverage which seems good for minimal head movement. Using 0.75 and 120 is fun for cinematic movies and standing up as it puts you more in the movie (my wife loved this).

Good to know that the cylinder and dome will both have the option to increase the radius in a later release. For 360 videos and images it probably won't matter but for limited coverage like I'm using for movies it will really help with the distortion.

I think it's time I donate to the cause. Can't wait for other options (3d subtitles, audio control/channels, better interface with VLC, etc.)
User avatar
goodl
One Eyed Hopeful
Posts: 19
Joined: Mon Jul 08, 2013 7:51 am

Re: A simple Virtual Reality media player project

Post by goodl »

If there was ever a project worthy of donation it is this one, I use it quite literally every day
User avatar
SinSilla
Two Eyed Hopeful
Posts: 71
Joined: Wed Feb 04, 2009 5:45 am

Re: A simple Virtual Reality media player project

Post by SinSilla »

Is there any chance that we will see a version of VRPlayer for android devices or is there any alternative for SBS content with distortion correction? With the upcoming release of Vrase, Durovis Dive or the already available OpenLoop it would be highly appreciated to have a proper 3d video player for these devices. While streaming via splashtop is a viable option at home a standalone player would still be needed for all other occasions.
am1001
One Eyed Hopeful
Posts: 4
Joined: Sat Apr 27, 2013 4:52 am

Re: A simple Virtual Reality media player project

Post by am1001 »

3c25030v.jpg
vrplayer stereograph.jpg
Hi

Great program.

One interest of mine is viewing stereographs (stereoviews) from the Victorian era and I have been testing a few in the oculus rift. There are 1000s of very high res ones on the library of congress site

http://www.loc.gov/pictures/search/?st=grid&co=stereo

I tried loading tiff files (very big!) and jpg and they do work in vrplayer and I can see 3d in the images.

However the vertical dimension is a bit squashed. I could not see a 'vertical stretch' feature in vrplayer.

Is that something that could be added? Also a horizontal stretch/squash in case needed also to make the image appear in the correct aspect ratio in the oculus rift.

I really like how it fills almost all your field of view when looking at such images with vrplayer.

I have attached a screenshot of a stereoview imported into vrplayer - hope you can see that it is vertically squashed.

Also it would be great if you could move to the next file in the folder using shortcut keys - when I had loaded a picture into vrplayer and pressed the A and D keys it did not switch to the next file in the samples folder. Is this possible - then we can view lots of images in one session.

thanks
You do not have the required permissions to view the files attached to this post.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: A simple Virtual Reality media player project

Post by geekmaster »

am1001 wrote:... One interest of mine is viewing stereographs (stereoviews) from the Victorian era and I have been testing a few in the oculus rift. There are 1000s of very high res ones on the library of congress site...
I like this one:
Image
;)
relaxman
Binocular Vision CONFIRMED!
Posts: 204
Joined: Sat Sep 19, 2009 12:07 am
Location: Hungary
Contact:

Re: A simple Virtual Reality media player project

Post by relaxman »

Hi
I made a 360 panoramic picture, resolution is 10.000x5.000 pixel
and looks very good with pano2vr and a simple web browser.
However when i want to view it in vrplayer resolution is degraded,
i see jagged edges. Why? Vrplayer don't support such pictures?
Cercata
One Eyed Hopeful
Posts: 21
Joined: Fri Oct 17, 2014 1:50 am

Re: A simple Virtual Reality media player project

Post by Cercata »

The latest version is the 0.5.1 from January 2014? Or the main site moved ?

The android version is the 1.5.1 and is from October 2014, but it seems to have less options than the PC 0.5.1 ...
relaxman
Binocular Vision CONFIRMED!
Posts: 204
Joined: Sat Sep 19, 2009 12:07 am
Location: Hungary
Contact:

Re: A simple Virtual Reality media player project

Post by relaxman »

StephaneLX
are you here? :roll:
doberso
One Eyed Hopeful
Posts: 6
Joined: Wed Apr 08, 2020 6:53 am

Re: A simple Virtual Reality media player project

Post by doberso »

This is looking good! A good alternative to my virtual cinema.
But the example pattern you show at the bottom is pincushion distortion right?
Its not the barrel distortion?
Post Reply

Return to “Oculus VR”