Mirroring raw video to external window w/o distortion

The place for all discussion of the Oculus Rift compatible open source 3D drivers.
Post Reply
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Firstly, I gotta say thanks for making such a great piece of software! Made my Dark Souls 2 playthrough possible, and that was a blast.

Okay, now down to the nitty gritty. I've been Twitch streaming with the Rift primarily using Vireio and retrofitting old games for VR. I usually end up cropping the game window to show only the left eye and square off all corners of the barrel distortion. This does a decent job of displaying what I'm seeing, but the chromatic aberration and barrel distortion are still very distracting on stream.

I was wondering if there would be a way to mirror the raw video from a game to an external window BEFORE Vireo's post-processing was applied. I could then display that window in my streaming software while the Rift displays the post-processed version of the same image. I think the external window would be useful for lots of demonstration purposes.

Any help would be much appreciated.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Mirroring raw video to external window w/o distortion

Post by cybereality »

Wow! Didn't realize you used Vireio for that.

In terms of mirroring the original feed, I seems like it should be possible but I haven't looked at the code in quite some time.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Yep, Vireio is my jam haha. Worked flawlessly the whole run.

I'm trying some things out, and it seems that Vireio will display an unaltered, non-distorted video into Open Broadcast Software up until the "CTRL+R to calibrate" function is used. The second you do CTRL+R, the image is split into L/R and all the post-processing is applied. It seems like it's so close to working properly. Is there a way to dismiss the calibration alert without hitting CTRL+R?
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

Hi

Thanks for the publicity your Dark Souls 2 playthrough gave Vireio. I believe Baggyg has added proper support for Dark Souls 2 to the "to do list", as it seems you played the whole game withou stereo separation. If we can add proper support you#ll get proper stereo, positional tracking, and VRBoost integration (giving 1-to-1 head tracking), which will make it even better, you#ll have to play the whole game through again :)

As for the mirroring thing.. believe me I have sunk many hours into trying to get a desktop mirror window to work, but due to limitations in DirectX 9, it appears it just isn#t possible (or is at least beyond my expertise). If you look at the github repo history and go back a few months you#ll see many aborted attempts to get it going. It is a shame as it would have been a very useful feature and it really hurt throwing the code away, but it just wasn't going to happen. So I'm afraid that at the moment it just isn't possible.

Finally, chrom aberration and distortion are applied from the start of the game, it doesn't have anything to do with the CTRL+R calibration, however if you would like to try it, you could set notifications="0" in he config.xml file, but that will turn off ALL vireio on-screen prompts, so you'll be flying blind.

I may revisit the mirror window one day if I have an inspiration, but for the time being, you'll have to continue using OBS (I actually tried some of the OBS codebase code to do it o, but to no avail)
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Anything I can do to help. I think what Vireio is doing is absolutely necessary for the success of VR. For me personally, the idea of retrofitting games for VR is just as appealing as new "made for VR" titles.

I'll have to make a video or something explaining what I'm talking about, but I'm almost sure it has something to do with the CTRL+R thing. It would only apply the distorted effect to the window capture after I used CTRL+R and not before, yet the Rift would still display it properly. I'm going to be streaming Mirror's Edge today, which is what I was trying it out with yesterday, so I'll get some more testing in there just to be sure. Thanks for the tip about removing the notifications! And please let me know if there's anything more I can do to help out.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

https://www.youtube.com/watch?v=dJYBS9B0QdI

So here is a video with the notification thing I was talking about. If I DISABLE notifications in config.xml, then it will start the game with L/R splitting and distortion. But when I leave them enabled and start the game, it will display properly in the Rift and in OBS as long as the notification remains on screen. As soon as I use CTRL+R and it goes away, it's back to L/R splitting. Hoping something good comes of it.

EDIT: I couldn't get the <youtube> thing working in the post, so I just put up a normal link.
EDIT 2: More findings here: https://www.youtube.com/watch?v=l9L9C1Lr9Ds
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

Wow, that's really odd! Thanks for the video, now I understand what you mean. I must admit I can't explain it. Just curious, if you open the vireio settings menu does it also go undistorted as well?

I'm afraid I can't currently explain that, it must be something to do with which back buffer on the swapchain that OBS is capturing from and it switching to the non distorted one when a notification is visible. Not something we can really fix in Vireio, as what it is doing is unexpected behaviour.

I could enable some sort of permanent tiny notification in an invisible part of the screen, to force this strange behaviour, but as notifications appear to affect fps this would really be a hack.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Did you check out the second video as well? It's gotta be something to do with when the positional tracking engages. So am I correct in assuming that CTRL+R calibrates your positional tracking doesn't really affect other head tracking? If positional tracking isn't being used, it seems like that's when the video displays normally in OBS. Could be something there.
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

The second video was the more revealing one really.. it isn't specifically the positional tracking itself that is anything to do with it I think.. it is the fact that when you move the HMD out of he tracking cone, you get the notification on the screen telling you the tracking has been lost. I think it is that notification that is then causing the OBS undistorted view effect.

The fact it is happening at all is purely accidental and I think i is a quirk of he way Vireio works (with stereo back buffers, one for each eye, plus the main render target which is the distorted stereo view), some funny DX9 stuff going on, and the way OBS captures. In this case the result is actually desirable, the only way to force this behaviour to occur so you can make use of it is to add a bit of a hack so that there is always a notification displayed (it could be in the upper left corner of the screen), that way OBS will always capture one of the undistorted buffers.

Could you run a few other tests please?
- What does the capture show when the FPS counter is visible (SHIFT+F)
- What does the capture show when the BRASSA menu is showing (CTRL+Q)
- Does it do this for any other games? (preferably a non-unreal game)

I could easily add something so you can switch this behaviour on, but will hide it in config.xml file (which people rarely touch) so no-one turns it on by mistake, as it will probably result in reduced FPS for some games.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

All right, so I gave it a go with Deus Ex: Human Revolution (non-Unreal Engine game), and it displayed as a distorted image in OBS. However, something weird was happening with my mouse where it wouldn't respond, so I opened up profiles.xml and deleted the info for Deus Ex and started from scratch, only inputting the game name .exe and Deux Ex as the profile's starting point.

Started Deus Ex up again, and voilà. Non-distorted image in OBS. It did the same thing that Mirror's Edge did where it would display normally until I hit CTRL+R and then it would distort/split. So I did some more testing and it turns out it absolutely is the notifications that are making the non-distorted video display in OBS. The BRASSA menu and the FPS counter both cause the image to "normalize".

What I'm wondering is what is it that I removed in profiles.xml that caused that effect to come back into play? With the default Deus Ex profile that comes with Vireio, there was no way to get a non-distorted image. Then, after I deleted all the profile info and started with the basics, it was able to display normally in OBS.

I would really appreciate the addition of the config.xml tweak ASAP. That would really make the stream look much better.
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

You may have inadvertantly changed the game type when recreating the profile entry, I suspect that would do it.

I'll add something in, will chuck you a binary to test over the next few days if I get time to do it. Bear in mind this is essentially a hack to use unexpected behaviour, so if it starts behaving strangely there won't be a lot we can do about it.

Also, if you are planning on playing Deus Ex next, then I'd recommend holding off for a bit as we are going to add proper support for that soon.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

I'll need to read up a bit on what the gametypes do. That'd be great if you could send me a fixed up version for streaming. I'd like to do more to help, but programming just isn't my cup o' tea. Luckily, tinkering is.

I guess I'll hold off a bit on Deus Ex then. I eventually wanna get Outlast looking right in it as well, but one thing at a time.
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

We're also just wrapping up Outlast support too.. that is also definitely one to wait for.. it will have a proper VRBoost profile for really decent head tracking and proper stereo.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Sometimes timing is on my side. I've been wanting to do that one for a while.

Let me know about those tweaked drivers when you get a chance. Thanks in advance!
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

I couldn't reproduce this behaviour myself. I set OBS to capture from the Game (I tried Bulletstorm and I think Mirror's Edge as well) but I got no undistorted capture, it was always the standard separated Rift view, irrespective of whether there were notifications on screen.

Can you describe how you set up the capturing please? as I suspect there is something I am missing.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

My OBS settings are very straightforward. It captures the game window, and that's it.

You might wanna try creating a null profile for the game in profiles.xml to recreate the effect, just using a profile name and a .exe entry. That's what triggered it when I did it with Mirror's Edge.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Any word on recreating the effect? Does the new Vireio release have anything special that could be helpful?
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

Not yet I'm afraid.. when I tried I couldn't recreate it.. will have another go and let you know.

EDIT: OK, I recreated it...

You can use LCTRL+LALT+Cursor Up to switch when in the render pipeline the notifications are rendered, and it seems one of them wil cause this behaviour to occur. The advantage of switching it like this is you don't need to strip everything off the profile, so you still get stereo separation and VRBoost etc, I'd advice doing this rather than deleting stuff from the profiles.xml.

Now I can reproduce it I can put some small back door in so that some unobtrusive notification is rendered all the time which should cause this behaviour to occur, hopefully not too hard to do.
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Excellent news! I didn't want to sacrifice stereo separation/VRboost, but it was much better for the overall viewing quality in OBS. Hopefully that backdoor works out and it will mirror clean video soon.

So are you going to put like a notification containing a ' at the top left corner of the screen or what? I'm curious about the method you're going to use for that.
User avatar
DrBeef
Vireio Perception Developer
Vireio Perception Developer
Posts: 328
Joined: Wed Aug 14, 2013 6:20 am
Location: London

Re: Mirroring raw video to external window w/o distortion

Post by DrBeef »

You're correct, that's exactly what I was going to do :)
dirtythekid
One Eyed Hopeful
Posts: 16
Joined: Wed Jul 04, 2012 11:10 pm

Re: Mirroring raw video to external window w/o distortion

Post by dirtythekid »

Damn I'm good 8-) . I can't wait to try this, please let me know when it's implemented ASAP as possible.

That's right, as soon as possible as possible.
hoopera
One Eyed Hopeful
Posts: 6
Joined: Thu Apr 17, 2014 4:33 pm

Re: Mirroring raw video to external window w/o distortion

Post by hoopera »

dirtythekid and Vireio: Making VR YouTubers' lives easier, one backdoor at a time!
User avatar
Losti
Golden Eyed Wiseman! (or woman!)
Posts: 1545
Joined: Tue Sep 17, 2019 5:30 am

Re: Mirroring raw video to external window w/o distortion

Post by Losti »

deleted
Post Reply

Return to “Development / General Discussion”