I might have rifted all sources like Nvidia/Tridef/IZ3D

[noos]

Hello,



I use video processors to alter video signals - scaling, framerate conversions/double and triple flashing, colour space conversions, 3D format conversion, deinterlacing, colour/brightness/contrast/gamma correction, aspect ratio control/zoom/cropping, softedge blending, geometry corrections via warp map and - very important - motion interpolation. But with every new format or industry standard, a stand alone video processors becomes outdated. And they are limited to what they were intended for, so only a few functions from above per a very expensive device.
So my idea was to set up a video processor for the Rift, that is based on a low level PC (equipped with a Blackmagic Intensity Pro Card), and is capable of applying the following (in real time) to any external source. A computer that can do that to a source is for simplicity named Rift PC in the further text:

- the geometric correction needed by the rift

- scaling to the resolution/refreshrate accepted by the Rift

- convert 2D/3D formats to the side by side format of the Rift

- colour/brightness/contrast/gamma correction

- aspect ratio control/zoom/cropping

- motion interpolation (the software I use so far - Stereoscopic Player - can easily integrate the "Smooth Video Project" expansion and apply motion interpolation to video files. Until now, I was not able to achieve this with external inputs. But future software solutions, or updates to recent ones should enable motion interpolation for external sources. But not at the cost of more latency - so effective interpolation modes are needed.

Thanks for reading so far. For your patients, and to motivate you to continue reading, I add a small video here. It shows the low latency with a Xbox 360 being rifted - on the fly/live/realtime.2D HDMI XBOX works directly with HDMI ports of a Blackmagic cards, as a 2d XBOX signal is not HDCP protected:


XBOX 360 latency test while realtime rifted for Oculus Rift (e5200 WinXP):
http://www.youtube.com/watch?v=FurYs2w98Po

Edit: New video XBOX 360 latency test while realtime rifted for Oculus Rift (Phenom X6 Win7). Read description in post 8 below - no detectable latency:
http://www.youtube.com/watch?v=ewdNoYq7MfM

There is a small latency, but please consider what is done to the 720p60 output of the 360: Scaling to 1280x800, geometry correction, and (if needed) 2d(3d) format conversion to side by side. More important is that the software (Stereoscopic Player) was never optimized for low latency, nor is a standard Win XP computer I used as Rift PC. I run a not loaded XP system in conjunction with an E5200, 2gb 800mhz RAM and a Nvidia 7200. Why so lame? Because a Windows 7 computer based on a recent Intel Quad at 4ghz, 2400mhz Trident RAM and a 7-Series GTX (thanks Matthias for your water-cooled beast), because this computer ended up with a huge delay on the real time playback. Both Rift computers had no firewall installed, nor where loaded with updates or software. But my XP based capture PC, after disabling GPU scaling and a few other things, was by far faster. So a "2400mhz RAM" Rift computer, with a optimized/lean operating system and an optimized software solution should give an even better latency.

Talking about latency: It can be considered general knowledge, that a "cable" controller gives a better latency in games than a cordless controller. So I wanted to find out if a cordless controller uses an "USB recharge cable" for communication, if the cable is connected. Therefor I used I a high frequency detector as it will visualize any emissions. Surprise surprise surprise: You will see in the small clips, that the PS3 Controller stops emitting when the USB cable is attached, while the XBOX 360 controller continues to communicate via bluetooth. But but but, that was not the big surprise. The surprise was that both consoles, even if wireless lan was deactivated, where emitting at full strength wlan radiation. Even if the console are in standby! "Always on" - somehow in this console generation too.

Xbox 360 emission test - wireless lan always on:

http://www.youtube.com/edit?ns=1&video_id=Pb9lM5LqUUA

XBOX 360 controller bluetooth emission test for latency optimization:

http://www.youtube.com/watch?v=aCQ84Z-w0WU

PS3 controller bluetooth emission test for latency optimization and wireless lan always on:

http://www.youtube.com/watch?v=AxfGTCaH7lQ


Back to the Rift PC. I decided to use a Blackmagic Intensity Pro PCIX card as capture card for the public project, because it is cheap and available for most users. It accepts 1280/720 at 50HZ/59,94HZ/60HZ, 1080p at 23,94hz/24hz/30hz, 1080i at 50hz/59,94hz/60hz, PAL/NTSC ans their progressive versions. Most important is that the card is supported (like newer/better Blackmagic cards) by the Steroscopic Player. Below are a few diagrams of how to playback sources in real time on an Rift PC. The HDMI splitter in the beginning is not an must, but it must work without delay. The Blackmagic produces "black screens" if a supported resolutions comes in unsupported timings. So with the HDMI splitter you can add a control monitor and always see your source. Even if Blackmagic works as intended, you will not need to wear your Rift for configuration.
The blackmagic will capture video/svideo/component at the above mentioned resolutions. Component can be converted to Euro Scart RGB via a transcoder - this conversion is lossless because it is a colour space conversion. So the Rift PC will accept all those mentioned sources. I will not go into detail with old SD and HD sources, but directly step to stereoscopic input of NVidia, Tridef and IZ3d.
The input of Tridef/IZ3D PC´s can go directy to the Rift PC and it´s Blackmagic with 720p60 SBS! 60fps per eye today via HDMI! Well, if you source PC can deliver them to the Rift PC. So you render the game including head tracking on the source PC - and then capture it and realtime rift it to the format of the Oculus Rift on your head.

Tridef/IZ3D stereoscopic 3d 720p60SBS as source - HDMI Splitter - HDMI to Blackmagic Intensity PCIx - Stereoscopic Player/Rift PC - output at 1280x800p6ohz "Side by Side" with Rift geometry.


Nvidia 3D is a little bit more tricky. Again you can directly connect the Nvidia source PC to the Rift PC and capture frame sequential at 720p60 (30hz per eye and depending on your PC also 30 frames per eye) via HDMI. Or a capture card capable of 120hz or a Lumagen Radiance....

Videos:

I hesitated to link videos captured with the Rift PC because of several reasons. First, I do not have a Rift and can not adjust 3D settings and the Stereoscopic Player. I even found a way to increase the low separation in PS3 stereoscopic games, a good deal more than you can see in Uncharted. But that is of no value without a Rift for control. So when you watch the videos, I need feedback if what I did results in a correct Rift format - I can not adapt it to comfortable viewing. My second reason to hesitate with public videos is that this Rift PC is optimized for realtime playback. As you see in the camera video, playback is smooth - even at 60hz frame sequential. But when I capture with the lame but low latency Rift PC, the videos do stutter. So please keep that in mind. I even was forced to deactivate sound recording to unlock capacities.


Thanks again to Matthias for the helping camera hand, and his massive XBOX collection.


NVIDIA stereoscopic test application 720p60 framesequential realtime rifted for Oculus Rift
http://www.youtube.com/watch?v=r6GUXpwmA4M
http://www.youtube.com/watch?v=CTIk08Zjsg0
http://www.youtube.com/watch?v=5_0Pg1NJp5Q
http://www.youtube.com/watch?v=PmLkHv7-Xk0

Tridef does Trine and then realtime rifted for Oculus Rift
http://www.youtube.com/watch?v=HlMsEqIzs0w

Nvidia does Dreamfall and then is realtime rifted for Oculus Rift
http://www.youtube.com/watch?v=69obWIdTQFI
http://www.youtube.com/watch?v=C1HgCBb-96g


IZ3D does Trine and then realtime rifted for Oculus Rift
http://www.youtube.com/watch?v=kYpwg7xw7Uo


PS3 and Xbox 360 3D (!) sereoscopic (!) gaming on the Oculus Rift:
https://www.youtube.com/watch?v=vptzBX4YMsM
https://www.youtube.com/watch?v=MGgeCOZZrPk
https://www.youtube.com/watch?v=WnwN2rEDqcc
https://www.youtube.com/watch?v=VArT1wtQ5pU
https://www.youtube.com/watch?v=VdUbixsbt58
https://www.youtube.com/watch?v=GBsg4wFA0cM
https://www.youtube.com/watch?v=wZPEokY_q1A
https://www.youtube.com/watch?v=igwEqCgN-FU
https://www.youtube.com/watch?v=XOX0J0wHSUY


Two points are left:

First - to the person giving out free Oculus Rifts to researching persons:
I could need a Rift for improving this solutions and realizing others. Also, as soon as HD-Rift is available, I could adapt the real time rifting solutions to the new panel.

Second - to the staff manager of "Team Oculus Dallas":
Consider this project the attachment of my application. My Job is to identify problems in all (hardware/physical) fields, become an expert and solve them - including the writing of unassailable patents (if patents are possible or necessary.)


Thanks for reading and best regards from Munich, Germany



Marc





Edit:

How to realtime rift Tridef/IZ3D, other Side by Side sources and 2D ( Oculus Rift )
https://www.youtube.com/watch?v=ZvwM0EqX6pU

How to realtime rift Nividia ( Oculus Rift )
https://www.youtube.com/watch?v=EO6FPXDcjOM

Just some art in the dark (Oculus Rift )
https://www.youtube.com/watch?v=XfeqhWpar_8

[ss248]

Intel Quad at 4ghz, 2400mhz Trident RAM and a 7-Series GTX

10/10 Would laugh again.
Thank you.

[JoeReMi]

Intel Quad at 4ghz, 2400mhz Trident RAM and a 7-Series GTX

10/10 Would laugh again.
Thank you.

Why?

[urban]

Great if this is actually true, what amount of latency do you think there is?

[cybereality]

Interesting technique. Watched some of the videos, looks promising.

[zalo]

Intel Quad at 4ghz, 2400mhz Trident RAM and a 7-Series GTX

10/10 Would laugh again.
Thank you.

Why?

I suspect he's laughing because that doesn't really tell us anything.

Quad Core @ 4 ghz? How old is it? There's a lot more to processors than clock speed and number of cores (if the program can even utilize all the cores effectively).

2400mhz ram? How much?! 512mb?

7 series gpu? There's a big difference between the GTX 760 and the GTX 780.

OT: Any hard numbers for latency or FPS? Or do you need the latency tester?

[WiredEarp]

I suspect he's laughing because that doesn't really tell us anything.
Quad Core @ 4 ghz? How old is it? There's a lot more to processors than clock speed and number of cores (if the program can even utilize all the cores effectively).
2400mhz ram? How much?! 512mb?
7 series gpu? There's a big difference between the GTX 760 and the GTX 780.

It tells me quite a bit really.

Most likely, hes talking about an i5, 3400K, overclocked to 4Ghz. The amount of RAM would most likely be at least 4GB, and a 7 series GPU shows that it has a modern enough video card, regardless of its particular model number, unless hes getting something under a 760 which I doubt.

So, basically, hes saying a decent CPU and a mid level video card at least.

[noos]

Hello,

and thanks for your interest.
It is not about processing power as I tried to show with the comparison. Real time playback without capturing left a lot of cpu capacity unused - with the e5200! Gpu processing power is not needed, so I did not mention what kind of cards were tested. It is the latency of the operating the system or the GPU drivers/options. That was identified after a dozen attempts if bad latency is present. For weeks I tested different hardware configurations, most of them very new. I also did new installs of the different versions of Win7. But one point that puzzles is, that nearly all Win7 systems had a perfect latency in a windowed playback mode, and than a useless latency while fullscreen. So this gives an idea where the fault is.

...just finished one more test - Phenom X6, 4gb 800mhz ram, GTX280, fully updated WIN7 Home with dozens of programs installed, fresh installed 326,41 beta Forceware - fullscreen:
Watch the left side of the right screen. Again, focus the right screen and it´s left edge carefully - concentrate on when the motion stops. Also peer hard on the Rift screen´s right side and let your eyes detect the motion, not any details. My eyes were not able to detect any difference when the motion stops on both screen. That means I was not able to detect a latency with the eyes. Hardware certainly will be able to detect latency. I want to add that switching through the menu is super smooth is realtime. The 30fps camera video can not catch the 60fps (or 30fps ?) of the XBOX menu on both screens.


XBOX 360 latency test while realtime rifted for Oculus Rift (Phenom X6 Win7)
http://www.youtube.com/watch?v=ewdNoYq7 ... e=youtu.be



Now I am getting more and more confident that we get all your computers to a low/no latency via this thread. Just post your system when you encounter a bad latency, and we will go through all points that helped here. Some of it can only be psychosomatic, as the single display performance mode in "3D" Forcware options and the activation of the performance mode, or to remove Nvidia Experience from Autostart. GPU scaling did not make a difference in my last setup.
But what helps in all cases is a must: When you change input format f.e. from SBS to framesequential in the Stereoscopic Player, restart Stereoscopic Player. If not, you will have a bad latency. When you restart the Stereoscopic Player, the last active mode will be used. Changing or activating a different input source does not have a negative effect on latency.
With the X6 running at low latency too, I can now rift any game and capture it while low latency playback . Some long wished games to be rifted in stereoscopic?

Regards


Marc


Ps.: All of you that want to reproduce what I did, should start with 720p60 SBS material from Tridef or IZ3D (you will have 60fps per eye). Or you use the Nvidia drivers at 720p60 framesequential (30hz/fps per eye getting double flashed - this is not motion interpolation!). Or some of you can afford better capture cards for 720p120hz framesquential, or Frame Packing for Nvidia 60hz per eye playback. But the plain Intensity Pro card is a good start, and then wait for the Fury to output full 60 frames SBS formats. Cards that accept higher inputs, do multiply the costs - so I excuded them.
....very much fun to start is also Gears of War 3 for XBOX 360. It outputs 720p60 SBS stereoscopic unprotected. So you can directly capure it with you Blackmagic and rift it

[noos]

...and WiredEarp really nailed the specs.
...and to urban: Sorry, can not measure latency here.

[noos]

Just sth. I want to share, because I think that could bring you the benefits of (PC-)videoprocessors closer.

On the source PC I was converting 2d files via live playback in WinDVD and PowerDVD to 3D. The output was Nvida shutter mode and captured by the Rift PC and realime rifted to Rift format. So what ever works on any known 3d output, can now be rifted.
Do you think some "how to" videos could help to bring you playback via realtime capture closer? I was thinking about one video per source.

Regards


Marc

[amermahdy70]

great thinking ,this might be the future, no need to look for rift compatibility from big game titles, everything will be rifted , it will be great if this can be developed to use an android box not PC so it can be small and cheap as drive box for the rift.

[noos]

Good thoughts!
By now I have used it with really small computers, just a little bigger than a DVD box. All you need is two cores and a PCIX 1x port.

[noos]

I will soon add 720120hz, 1080p60 SBS and 4k support
You only need better catpure cards, thats all.

[mr.uu]

!NO! - just ordered a Blackmagic Intensity Pro to try what you are doing
I have an old XBOX360 laying around.
Which card you want to use as an upgrade?

[noos]

Relax...

Upgrade cards do cost a multiple of a Intensity pro PCIx. That is why I used it, because I wanted as many people as possible to be able to reproduce what I am doing. The Intensity is the best/perfect match to the current Devkit display. Anything else means burning money.
I am in contact with Blackmagic, and I will recomend other cards only after I have tested them for full compatibility in the Rift PC.

[baggyg]

Hi noos,

Thanks for an interesting topic. From the videos it looks very good with a low latency however I must say it doesn't look ultra low latency and potentially not low enough for the Rift. I commend your efforts but think you really need to measure the latency to convince people. Even a perceived low latency could be too much whilst using the rift leading to a lot of sickness.

As I said keep up the good work.

[noos]

Thank you very much for the feedback. Anything is very much appreciated!
All of the videos were produced with the e5200 PC. This PC showed a low detectable latency. The X6 based PC with 800MHZ ram (and it´s latency video) was the first to show a not by the eye´s detectable latency. Or did you see one? Now, with a recent PC and fast RAM it should be even better.

But let skip head tracking for a moment. What about just using the ghosting frees 3D abilities of a HMD, and abuse it sometimes as plain but premium large sceen device - ghosting free. In some scenarios, like gaming marathons, this might have advantages. Do not get me wrong, I know about the impact of headtracking as the biggest part of the VR experience. I know that because this is missing in my "walk into" curve screen stereoscopic projector setups. This is my field.

Tomorrow in the morning I will shoot the how to videos. They should be online and linked to the forum in the afternoon CET. I am sure the first Blackmagic Intensity cards arrive soon at the doors of some pioneers, so I am looking forward to their experiences and improvements.

[noos]

Upated first post with "how to´s and art

[tomstereo3d]

Hello,

the owner of the site "www.bloculus.de/" had the idea to run and rift PC games on one PC only. So using the capture card in the same PC that is running tha game. Is that possible?

Thanks, and great work! I am looking forward to play Playstation 4 with your solution on the Rift!

Tom

[baggyg]

Hello,

the owner of the site "www.bloculus.de/" had the idea to run and rift PC games on one PC only. So using the capture card in the same PC that is running tha game. Is that possible?

Thanks, and great work! I am looking forward to play Playstation 4 with your solution on the Rift!

Tom

Why would you want to? Surely using Overlay or Deskope would be more efficient if Tridef, Virieo or VorpX are supporting the game (whcih by the time VorpX is out will have a lot of games covered).

[tomstereo3d]

For Nvidia/Nvidia Quadro (CAD) 3D. Or running old opengl Games via the classic froceware 3D that was limited up to 7series GPUs - anyone tried Prey and Quake 4 on a Rift ?