Vk3DVision - A Vulkan 3D Vision Driver (With Single-Frame Stereo and RTX Support)

[for3donly]

phemomenal ! it's a great milestone. love it to see vulkan 3d vision unleashed like this. thanks Helifax.

[helifax]

I made a Patreon Page for this project: https://www.patreon.com/Vk3DVision
I also made an official page for the project: http://3dsurroundgaming.com/Vk3DVision.html

I know a few people donated generously to this project already using Paypal (MANY MANY THANKS! This was not foreseen!)

I have opened a Patreon Page for the project, simply because I am curious to see how may people would actually support such an effort!
Sure, FREE software is always great! We as CONSUMERS can testify for that! Both Helix wrapper (Dx9) and 3DMigoto (DX11) being free is great for having for free and for being maintained by DarkStarSword, Bo3d and let's not forget for Flugan's DXBC decompiler additions! Not to mentioned of all the 900+ fixes made ON TOP of these FREE software wrappers!
I used them on numerous occasions for all my DX11 fixes that I made (with the community's help!)

I know for HelixVision, you pay a literally one QUALITY BEER price (at lest here), and I would pay 4x the price for that software, if asked!!!

I have released all my OpenGL3DVision wrappers for FREE & all the OGL fixes as well!
I have released my "Compatibly Mode Unleashed" Mod for FREE, as well!
I know, not many where interested in my OGL wrapper at the time, as most games where Directx9/11 games - where we could leverage on 3D Vision Automatic, to do the work for us! In my OpenGL wrapper , well, you need to do all the work!

Sadly, Nvidia 3D Visiom.....is dead.... even if we like it or not!
But, Stereoscopic 3D Rendering is NOT as we can see it in HelixVision and Virtual Reality!

So, now, that DirectX 11 comes to AN END and Nvidia 3D Vision Automatic is at AN END, we need to look to the future! Not to mention that literately everyone is looking at VR for a stereo3D experience!
This project is basically looking at how to Stereo3D Rendering is possible for Vulkan games:)

So, if you think that all my previous OpenGL-3DVision wrapper and al my Stereo3D fixes SUCKED, then please ignore this post!

If, however, you believe, this is the time to go FORWARD then, please, consider this effort
I don't care about the Patreon Moneys (I can't ask for less then 5£/$/Euro)! I am more interested, in how many people are actually interested in effort and 2D to REAL Geometry 3D Stereo rendering

Cheers & stay safe!
Helifax

[john-glad]

Hey Helifax,

I just joined your patreon
I really like the idea of an Vulkan Stereo 3D Wrapper as 3dmigoto is fantastic but it seems there will maybe no DX12 implementation soon and this is the best way to go as alternative, as more and more games in the future will shift to either DX12 or Vulkan. Also although I love 3D Vision I feel that the upcoming generation will suffer even more from the 3-core bug and I hope that this could maybe also be a solution for this problem. So I wish you the best for your project and hope that this will work out somehow. Thank you very much for your effort and best luck.
And BTW would this new approach also work on 4K EDID TV with 60HZ or only on 120Hz Monitors like the old OGL wrapper?
Greetings

[Losti]

Hey Heli, joind you on pateron !

[thebigdogma]

I went old-school on PP!

[helifax]

Many thanks to everyone that clearly showed they are invested in this project and are determined to make Stereo3D rendering happening in Vulkan!
I did not believe I would get such a reaction!
Many thanks guys it is much appreciated!

Cheers,
Helifax

[helifax]

And BTW would this new approach also work on 4K EDID TV with 60HZ or only on 120Hz Monitors like the old OGL wrapper?

I updated my OpenGL wrapper (version 10.1) some time ago to also support Stereo3D Rendering in both SBS and TU formats without the requirements to use 3D Vision. (It's like ROTTR and SOTR do it).
The requirement to have an Nvidia GPU is still there as I am still using the NV DX-OGL interop. I never found a reason to change it, but in theory I could remove this proprietary extension as well so it would work on AMD gpus as well.

You should be able to grab any of the fixes using 10.1 version of the wrapper and enable the alternative rendering methods in the ini file.
Hope this helps!

Cheers,
Helifax

[john-glad]

And BTW would this new approach also work on 4K EDID TV with 60HZ or only on 120Hz Monitors like the old OGL wrapper?

I updated my OpenGL wrapper (version 10.1) some time ago to also support Stereo3D Rendering in both SBS and TU formats without the requirements to use 3D Vision. (It's like ROTTR and SOTR do it).

Oh great. I missed this update. Thx for the info. I will then maybe have look again and try it out.
Great progress and wish you the best
Greetings

[Skawen]

Joined patreon too

[Necropants]

Wow, that's all I have to say at this point. We are truly blessed to have people like Helifax still fighting the good fight even as expert shader hackers continue to dwindle.
I do not like the idea of Patreon because I always forget my expenditure for subscriptions based model stuff. but I will be donating directly when I can.

By the way It's not a game I am interested and this is not the right thread for discussion but Deaths Stranding has been confirmed Direct X 12 only. I suspect next gen will see the move too dx12/vulkan for most new games.

[Lizzard]

Im glad you asked Helifax. Nobody in this community wants to see stereoscopic gaming die. As far as possible I keep donating to the fixes you guys release. If you shaders hackers never ask then you will never know. I understand that there was a "code" over when things were all pro-nVidia. But they've abandoned us. Not completely though cause I heard a bird tjurping that they are working on a VR headset. If they develop software that would be attractive to VR game developers then matters might sway in a weird goodish direction. It probably means more contend, steady growth and less work for others. But this remains to be seen on top of their streaming services. For VR contend, streaming services would be top notch! Since that rented hardware is not being used by the SAME person 24/7 it might be a profitable project. But in streaming they will be operating all the time with different users. Presumptuously the PC will be milked for performance and no unnecessary background processes running. A "store" where you would be able to experience people's apps and products. For 3D movies this would be a perfect platform. You dont need a powerful pc for a movie. Unfortunately tech giants are all fighting to get to the top. Where is the f#in top? Where mountains fight?

If I were you I would present this to individual game developers when you've achieved good results. How many game developers are here? Show them the possibilities. Let them help with funding. It might take a really big step. But as Bo3b reported, if I understood correctly, there were last counted 1200 users of HelixVision. Not all of them are from the old fans/supporters like us. HelixVision is still young and already offers a great deal of fun. Yes! The price is heavily undervalued for what it offers. I bought it without blinking an eye. Even before I had a VR headset. The guys who had 3d projectors knows what hazzle a home cinema entails. Also the immersion that comes with it. And the almost 100% crosstalk absence. But 720p sucks (ROTTR with VR was way better. 60% Gpu restricted 1080ti), which in contrast is slightly better than HMD screens. Denser textures was an enormous boost. Now that I've played with Oculus Rift S and Samsung O+, I have a better idea how these entry level experiences compare on a modest 7 year old system. Immersionwise, VR wins by miles! The Pro's outweighs the cons. And team Helix developed some really neat functions that took immersion to a whole new level because of its flexibility. There are plenty 1080ti's and stronger things out there now.

This might be stereoscopic gaming's last straw. If it doesnt happen now, it never will. I've never longed to go back to my monitor or big screen. You have to sacrifice the fidelity that high density DPI screens offers. But the SamsungO+ has a very near perfect anti-aliasing effect designed into it. It surpasses the Rift S. I would recommend it to any first comers. If you prefer better tracking, customisable headphones, and more screendoor then the Rift S would suffice. I personally liked the bigger FOV of the O+ more. They are both great as a package and first deal.

Not to get carried away. I just love this tech and everyone involved.

I wish all you binary chemists the best. Youve done a great job to help develop this tech that so many of us still love after so many years. Please never feel underappreciated. Let every man do what he can.
But lets not get emotional.

(Times are tough and our foundations are being tested. You need to know who or what you love in this life. Introspectively. Each to his own.)

Im still glad to see there are people who keeps going and wish that more people can experience what you guys made possible, to game developers, for gamers, at no charge. Every guy who appreciates this tech and wants to help it grow should pull some weight how ever they could. You shader hackers should have been famous already. haha I miss the big team you guys were once. This tech certainly came on a long journey! Even the pyramid concept was once marveled on worldwide. Now the war is who can design the finest pixel. If things were less war, more together, then perhaps more advance? Where is the f#in top?

[helifax]

A frame presented using Nvidia 3D Vision:


(Like I said before, the rendering is wrong, but in Stereo3D as seen)

Sadly, I am forced to use DCH and newer Nvidia Driver + 3DFM fix to get Nvidia 3D Vision to run on my laptop. Driver 425.31 (both standard and DCH) refuses to find my RTX 2080 in my laptop.
I noticed that using this approach I lost the ability to take 3D Screenshots
So, I recorded in Stereo3D (awesome that it still works) using ShadowPlay and I cut one frame which is the above picture:)

I was able to enable Exclusive Fullscreen in DX11 for 3D Vision to kick in, but sadly it's very fiddly and there are good and concrete reasons for this:
- Vulkan uses worker threads for the graphics drawing and creation. One one of this threads I detect when it makes the Vulkan device and create my own DX11 device.
- It uses 1 thread for rendering.
- Has another "Main Thread" for the window handling.

I cannot change to exclusive fullscreen from a worker thread. This needs to be done from the main thread! (Or else go down a synchronisation rabbit-hole)
I registered a hook for WndProc and I tried to set fullscreen from there, but for some reason, it switches to fullscreen and then reverts back to window mode. (I think some messages between DXGI and game window are responsible for this).

The current workaround is to SPAM ALT+ENTER until Exclusive Fullscreen "STICKS" and the game presents in 3D Vision ) (And you do need to spam it at least 5 times ^_^)
I'm still trying to see if I can't do it automatically somehow... The biggest difference between DX11 -> DX12 & Vulkan is DX11 was mostly single threaded, DX12 & Vulkan are very HEAVY Multi-threaded applications.
Oh, and with every resolution change, this games loves to destroy the old Device & Swapchain and make a new one. I need to do the same for DX11 obviously:D Guess what happens if you try to remove a device that is in Exclusive Fullscreen Mode

Unfortunately I cannot used DX9 (as I did in my OGL3DVision wrapper) for presenting, where Nvidia 3D Vision works in Window mode. (I require DX11 for Vulkan interop and DX9 to DX11 interop is very very basic. It only supports 3 formats which I am not using...) So, DX11 it is.

Hope you like it:)

Cheers,
Helifax

[lohan]

Couldn't believe my eyes this morning when I noticed you are back, helifax! ...and with a BANG...if I may say so! But it wouldn't be a proper "helifax-Comeback" if it didn't involve some kind of Herculean task, right!?

Since you always have been the FOV-herald on this forum and also became a VR-user in the meantime I will gladly share my thoughts on the Pimax 8KX as soon as it arrives on my doorstep (I am one of the first pre-orderers). I sold all prior Pimax headsets (5K Plus, 8K and 5K OLED) in favor of the Valve Index (although all those Pimax headsets were also great headsets) but I guess the 8KX will take the crown.

Sorry for the little side-note. Back to topic now!

[helifax]

I managed to finally crack the "nut" and fixed the Exclusive Mode requirement for Nvidia 3D Vision to render!
I made the wrapper render with Detroit: Become Human and Wolfenstein: Youngblood. (These are the only 2 games I tested so far!)

DXGI Fullscreen Exclusivity is something not to take lightly, especially in Multi-Threaded Enviroments, that are all Vulkan based games! This is because of the requirements for DXGI to switch to Exclusive Fullscreen mode and the rabbit hole is quite deep
The game was meant to render in Vulkan. I am basically adding a D3D11 Renderer on top for 3DVision and HelixVision and as such, there are lots of caveats

I made a quick video (using my phone) that shows that while using my Laptop (with the DCH driver and 3DFM + fix on Windows 10 1909) I launched Detroit in Exclusive Fullscreen mode (in DX11), Nvidia 3D Vision kicked in, the Nvidia 3D Vision emitter started and that the glasses where in perfect synchronisation with the monitor screen (and you can actually see the L & R eyes through the lenses).

https://www.youtube.com/watch?v=LoG8AsAysqY


This was harder then expected, to be host, but I am really happy about the progres

Hope you like the video and the progress so far!

Cheers,
Helifax

PS:
Wolfenstein: Youngblood has some issues with the shader decompilation, creation. Most likely bugs, that I will need to track, hence why I couldn't present a Stereo3D image

[bo3bber]

Sadly, I am forced to use DCH and newer Nvidia Driver + 3DFM fix to get Nvidia 3D Vision to run on my laptop. Driver 425.31 (both standard and DCH) refuses to find my RTX 2080 in my laptop.
I noticed that using this approach I lost the ability to take 3D Screenshots
So, I recorded in Stereo3D (awesome that it still works) using ShadowPlay and I cut one frame which is the above picture:)

I'm not certain, but it might be worth trying the latest HelixVision beta_driver branch for this.

The beta_driver will install a Schwing hacked video driver into the game directory, while leaving the system driver untouched. It works with DCH drivers, and has no impact when 3D Vision is disabled. It does require the special 3Dmigoto that ships with it, automatically also updated.

This might complicate your environment though, adding 3Dmigoto into the mix may or may not be helpful. 3Dmigoto also has a force full screen function that bypasses DXGI, that might work to avoid the alt-enter problem. Lots of moving parts.

I cannot change to exclusive fullscreen from a worker thread. This needs to be done from the main thread! (Or else go down a synchronisation rabbit-hole)
I registered a hook for WndProc and I tried to set fullscreen from there, but for some reason, it switches to fullscreen and then reverts back to window mode. (I think some messages between DXGI and game window are responsible for this).

Try using Discover mode instead. You can run a lot of DX11 games in windowed Discover mode, and still have 3D Vision enabled. I set up a bunch of games this way for HelixVision for people that don't have any 3D Vision hardware.

Discover mode is not what you want as output, but for HelixVision/VR/SBS presentation, this can work. I grab the backbuffer before the NVidia Present call, which is why inside VR it's not red/blue. You can possibly do something similar, not sure. With a second monitor for example, a game could run on the non-3D one windowed in Discover, and you present in 3D on the main one. Dunno, just thinking out loud.

I'll experiment with Detroit to see if I can get this working, as a data point.

[helifax]

Sadly, I am forced to use DCH and newer Nvidia Driver + 3DFM fix to get Nvidia 3D Vision to run on my laptop. Driver 425.31 (both standard and DCH) refuses to find my RTX 2080 in my laptop.
I noticed that using this approach I lost the ability to take 3D Screenshots
So, I recorded in Stereo3D (awesome that it still works) using ShadowPlay and I cut one frame which is the above picture:)

I'm not certain, but it might be worth trying the latest HelixVision beta_driver branch for this.

The beta_driver will install a Schwing hacked video driver into the game directory, while leaving the system driver untouched. It works with DCH drivers, and has no impact when 3D Vision is disabled. It does require the special 3Dmigoto that ships with it, automatically also updated.

This might complicate your environment though, adding 3Dmigoto into the mix may or may not be helpful. 3Dmigoto also has a force full screen function that bypasses DXGI, that might work to avoid the alt-enter problem. Lots of moving parts.

I cannot change to exclusive fullscreen from a worker thread. This needs to be done from the main thread! (Or else go down a synchronisation rabbit-hole)
I registered a hook for WndProc and I tried to set fullscreen from there, but for some reason, it switches to fullscreen and then reverts back to window mode. (I think some messages between DXGI and game window are responsible for this).

Try using Discover mode instead. You can run a lot of DX11 games in windowed Discover mode, and still have 3D Vision enabled. I set up a bunch of games this way for HelixVision for people that don't have any 3D Vision hardware.

Discover mode is not what you want as output, but for HelixVision/VR/SBS presentation, this can work. I grab the backbuffer before the NVidia Present call, which is why inside VR it's not red/blue. You can possibly do something similar, not sure. With a second monitor for example, a game could run on the non-3D one windowed in Discover, and you present in 3D on the main one. Dunno, just thinking out loud.

I'll experiment with Detroit to see if I can get this working, as a data point.

Hi Bo3b,

Getting 3DVision Discover to display is not an issue. Getting HelixVision to display is also not an issue.
The issue is getting to Display to Nvidia 3D Vision as it requires DXGI Exclusive Mode swapchains to enable the monitor, glasses and emitter
MSDN is pretty clear on why enabling DXGI Exclusive Fullscreen must be done on the same thread as the main Windows WndProc and why it dead-locks, if not
https://docs.microsoft.com/en-us/window ... guide-dxgi (I think I know this page by heart now )

Those are the fights I had to fight and win to get it working In the end, I had to synchronise between the RenderThread and the Thread of the Rendering Window (also Detroit, spawns another input that is used for Input, so I had to rule that one out as a rendering window )

I know this sounds a bit confusing without seeing the game & code in action

Cheers,
Helifax

[bo3bber]

Hi Bo3b,

Getting 3DVision Discover to display is not an issue. Getting HelixVision to display is also not an issue.
The issue is getting to Display to Nvidia 3D Vision as it requires DXGI Exclusive Mode swapchains to enable the monitor, glasses and emitter
MSDN is pretty clear on why enabling DXGI Exclusive Fullscreen must be done on the same thread as the main Windows WndProc and why it dead-locks, if not
https://docs.microsoft.com/en-us/window ... guide-dxgi (I think I know this page by heart now )

Those are the fights I had to fight and win to get it working In the end, I had to synchronise between the RenderThread and the Thread of the Rendering Window (also Detroit, spawns another input that is used for Input, so I had to rule that one out as a rendering window )

I know this sounds a bit confusing without seeing the game & code in action

Cheers,
Helifax

Ah, roger that. It's because you still want to be able to display on our antiquated 3D Vision hardware. And as a 3D Vision guy, I fully support your dedication to fighting the multi-threaded fight.


Not for now, but in a future HelixVision setup, we can do a direct buffer sharing, no 3Dmigoto required.

If you decide you need the driver override mechanism I've put into 3Dmigoto, I'm more than happy to share my code for you to add that Vk3DVision, to avoid needing a 3Dmigoto for these Vulkan games to override DCH drivers. I've not added this to the primary 3Dmigoto Github, because this doesn't really have anything to do with 3Dmigoto, it's just a handy spot for DX11 crashers. And it also requires external support to build a proper driver. We could conceivably move the driver hacking into 3Dmigoto however.

This was kind of tricky- had to hook an undocumented ntdll.dll function to get it to work. Our testing is looking good here, this is likely to become the default approach for 3DFM as well, to drop the driver files into game directories instead of a system wide hack. User experience will be good- just install whatever driver they want, and it will auto-hack the driver and setup so that it can run 3D. Will be more clear as we test.

[helifax]

Of course,
ID-Tech 6 & 7 are using special NV extensions that the SPIR-V to GLSL decompiler doesn't know about it:)

I started working my way through all these problems like:
- crashing when a shader fails to be re-compiled.
- reporting when a shader can't be decompiled properly and recompiled back, with meaningful errors and separate shader dumps for "failed" shaders. (The plan is to manually fix them and feed them back into the application).
- crashes in Update Descriptor Layouts, when trying to update a Layout that we didn't modify since we couldn't recompile the shader.
- fixing some semantics in the decompiler (I don''t want to really start working on the cross-spirv decompiler. I can add new semantics for built-in variables, but I try to maintain it up-to-date with the official source) Decompilers and compilers are really hard! Especially to a higher language like GLSL or HLSL.

- The next phase is to see how I can load (at run-time) the modified shaders and swap the ones created by the game at creation time. I also plan on adding the ability to load the already compiled shader blob (so we don't compile it at runtime) from disk. This should speed up things quite a lot!
(Not that is slow, but it can be a bit "hitchy" in places)

Cheers,
Helifax

[helifax]

Sorry, it seems I missed a few questions in the thread and I am trying to catch up on them:

Also although I love 3D Vision I feel that the upcoming generation will suffer even more from the 3-core bug and I hope that this could maybe also be a solution for this problem.

This will not suffer from the 3-code bug. That bug only happens in 3D Vision Automatic (the 3D Vision driver does all the Stereorization and is limited to 3 cores in usage). I do my stereorization manually and while there are a few "hot-spots" for lock-guards, I am aiming at the ends to have a full Multi-threaded approach so we don't require the lock-guards

Couldn't believe my eyes this morning when I noticed you are back, helifax! ...and with a BANG...if I may say so! But it wouldn't be a proper "helifax-Comeback" if it didn't involve some kind of Herculean task, right!?
Since you always have been the FOV-herald on this forum and also became a VR-user in the meantime I will gladly share my thoughts on the Pimax 8KX as soon as it arrives on my doorstep (I am one of the first pre-orderers). I sold all prior Pimax headsets (5K Plus, 8K and 5K OLED) in favor of the Valve Index (although all those Pimax headsets were also great headsets) but I guess the 8KX will take the crown.

Sorry for the little side-note. Back to topic now!

That is the idea!, If it's easy, then why should "I bother with it" ?
I am very interested in your Pixmax 8KX review/impressions! I'm still on my Vive Pro + lenses (I really like crystal clear pixels )

Ah, roger that. It's because you still want to be able to display on our antiquated 3D Vision hardware. And as a 3D Vision guy, I fully support your dedication to fighting the multi-threaded fight.

Not for now, but in a future HelixVision setup, we can do a direct buffer sharing, no 3Dmigoto required.

If you decide you need the driver override mechanism I've put into 3Dmigoto, I'm more than happy to share my code for you to add that Vk3DVision, to avoid needing a 3Dmigoto for these Vulkan games to override DCH drivers. I've not added this to the primary 3Dmigoto Github, because this doesn't really have anything to do with 3Dmigoto, it's just a handy spot for DX11 crashers. And it also requires external support to build a proper driver. We could conceivably move the driver hacking into 3Dmigoto however.

This was kind of tricky- had to hook an undocumented ntdll.dll function to get it to work. Our testing is looking good here, this is likely to become the default approach for 3DFM as well, to drop the driver files into game directories instead of a system wide hack. User experience will be good- just install whatever driver they want, and it will auto-hack the driver and setup so that it can run 3D. Will be more clear as we test.

Yes, that sounds like an awesome idea! I'll have to try it! Sadly, I wasn't been able: between working during the day and "hacking" on this wrapper during the evening/night

I DO have this idea in my head , which I haven't said it yet, but it's really simple:
- Since I am doing the Stereo3D in my code, I don't actually rely on 3D Vision for anything except FOR IT accepting 2 FULL frames and presenting them at once as 2 eyes.
- Since I am presenting sequential frames, I can have 2 buffers that I can send over to HelixVision and when it gets both of them, Present(). Then it clears both buffers, it gets L buffer, it gets R buffer and Present(). This is basically how I do the present in my wrapper.
(If in the future we modify the workflow to DUPLICATE draw calls and all associated Rendertargets & buffers) this would still work, since we will still need to say Activate_L <- Image, Activate_R <- Image and Present(). 3D Vision Direct I only use to display on a 3D Vision display, and I require the hack only for this and atm for displaying in VR using HelixVision as it only takes output from 3D Vision Automatic/Direct ^_^. But, since we both write the code, we could easily integrate this and make Vk3DVision work with HelixVision without 3DVision being a requirement )

At least for this project, this would remove the requirement of 3D Vision (and Nvidia Proprietary GPUs altogether. Yupp, it will work on AMD and even Intel GPUs - when they release)
What are your thoughts

Cheers,
Helifax

[skyrimer]

It's amazing to see this project move forward, thank you so much Helifax and bo3b, I just joined the Patreon to give some support to the project.

[helifax]

It's amazing to see this project move forward, thank you so much Helifax and bo3b, I just joined the Patreon to give some support to the project.

Many thanks for your support and for believing in this Project!

On a separate note, I was wondering, if we have an Art designer / Artist among us, that could, perhaps, create a better logo for this project (than my quickly put together one ^_^)?
I think Costiq is into art designing ?!

Cheers,
Helifax

[bo3bber]

I DO have this idea in my head , which I haven't said it yet, but it's really simple:
- Since I am doing the Stereo3D in my code, I don't actually rely on 3D Vision for anything except FOR IT accepting 2 FULL frames and presenting them at once as 2 eyes.
- Since I am presenting sequential frames, I can have 2 buffers that I can send over to HelixVision and when it gets both of them, Present(). Then it clears both buffers, it gets L buffer, it gets R buffer and Present(). This is basically how I do the present in my wrapper.
(If in the future we modify the workflow to DUPLICATE draw calls and all associated Rendertargets & buffers) this would still work, since we will still need to say Activate_L <- Image, Activate_R <- Image and Present(). 3D Vision Direct I only use to display on a 3D Vision display, and I require the hack only for this and atm for displaying in VR using HelixVision as it only takes output from 3D Vision Automatic/Direct ^_^. But, since we both write the code, we could easily integrate this and make Vk3DVision work with HelixVision without 3DVision being a requirement )

At least for this project, this would remove the requirement of 3D Vision (and Nvidia Proprietary GPUs altogether. Yupp, it will work on AMD and even Intel GPUs - when they release)
What are your thoughts

This should definitely be doable. Because we need 3Dmigoto to force-load the override/Schwing hack video driver, we need a special 3Dmigoto to do this for HelixVision. I've changed the version numbers to include a trailing build number, like 1.3.16.4 to distinguish these from official 3Dmigoto builds.

However, since we have this 3Dmigoto requirement, I also want to add a way for this 3Dmigoto to directly send the backbuffer to HelixVision for display. Right now 3Dmigoto hacks the game to load, and HelixVision also hacks the game to load. No reason to do it twice when 3Dmigoto already does a killer job, and can just publish what it knows.

My plan, not soon, couple of weeks at least, is to add a named pipe to 3Dmigoto so that it can tell Katanga about the backbuffer directly. This will make the launching and connecting to the DX11 games 100% reliable, because 3Dmigoto is already active. Will solve any weird timing issues with Katanga connecting.

Sort of a long explanation to get to- I'm thinking this would be the same for Vk3DVision. You'd just publish the backbuffer info via named pipe, and Katanga will pick it up and display.


Right now Katanga has a mechanism to determine if it's a 3D Direct game, and changes modes to handle it. In a normal 3D Vision Automatic game, the backbuffer is a 2x width buffer, presented at one time. Katanga grabs both eyes at once from that surface and using a special pixel shader drives the ouput to each Unity buffer necessary for 3D to be seen in VR as part of the Unity SinglePassStereo.

For a 3D Direct game, it's a normal single width backbuffer, and we need to call an nvapi->EyeSwap routine to choose which eye it contains. For 3D Direct games, the HelixVision hack in the game process grabs each eye in sequence, and writes them to a double width buffer which is passed to Katanga for display.

For the named pipe mechanism, we can do whatever winds up making the most sense. It's not hard to grab two sequentially, or alternatively, make a double width buffer. I don't think performance will be impacted either way.

It seems like doing one after another is the right choice for the time being, because you are going to with alternate eye rendering to start with. It's also worth noting that I have a hacked up DX9 project started that is doing the same thing. At some point I'll get back to that as a similar no-3D Vision necessary approach. And that will be alternate eye rendering as the fastest and simplest 3D to get working.

On a separate note, I was wondering, if we have an Art designer / Artist among us, that could, perhaps, create a better logo for this project (than my quickly put together one ^_^)?
I think Costiq is into art designing ?!

I think Costiq would definitely be interested. He did the HelixVision logo, and I gotta say I think it's pretty amazing and much appreciated.

Also tsaebeht made the original HelixModBlog logo and could probably help out.

[helifax]

Lots of tiny little things I worked in the last 2 days:
- I am now handling shader creation with care and I handle cases were the decompiler/compiler might fail to produce a shader.
- Started working on pipeline tracking mechanism (prototyping currently)

Instead of trying to fix Detroit blindly (modify a file, restart game -> see if it worked), I started working on the tooling that I will require for actually fixing the game:)
- I added all the UI from my OGL3DVision Wrapper for viewing all the shaders that are exported.
- I added UI for viewing the Shader Pair that are currently bound and responsible for rendering.
- Added the TextEditor (that allows in-game, real-time modification of the shaders)

A quick example of looking at the active shaders in Detroit's Main Menu:


Next part that I'll start working on, is Real-Time shader compilation and swapping as well as the ability to highlight the part in a frame a shader is responsible for drawing.

Hope you like it!

Cheers,
Helifax

[russellk]

Very impressive progress in a very short period of time Also thanks for the updates, you sound proud and so you should be!

[helifax]

Very impressive progress in a very short period of time Also thanks for the updates, you sound proud and so you should be!

I am not necessarily proud, but really happy that I am able to get some fast progress in certain areas of functionality! I aim to make it as easy to use as possible and to provide as much functionality as I can
I am still building all the blocks that I think I require to be able to fix, for example Detroit Once I''ll actually start fixing it, I'll probably see what else I need I've learned quite a lot from my OGL3DVision wrapper on the "do" and "don't" part

Really happy that you like it!

Cheers,
Helifax

[bo3bber]

Yes, extremely impressive progress. Fantastic to be able to reuse code from your OpenGL wrapper to make this faster. Immediate UI, awesome.


Tangentially related, but not exactly on topic- what is your opinion of gaming using Linux instead? Having a Vulkan wrapper would open up a lot of potential gaming there, and in 3D.

The reason I ask is Windows 10 has always been and continues to be a giant pain in my ass, and I'd personally love to be able to leave their incessant and worthless updates behind forever. Linux desktop is as good if not better than Windows 10 today, and you can at least stick with LTSB builds. Valve is still currently supporting VR on Linux (including Alyx), and Proton has received universally good marks.

[Ixomack]

Hi, just a quick word to tell you that I joined your patreon.
As a quiet lurker of this forum, i did not do much to help until now so it is time to change that and I hope other lurkers will take this opportunity to join the fight against the slow phasing out of 3D video games.

[Losti]

Hi, just a quick word to tell you that I joined your patreon.
As a quiet lurker of this forum, i did not do much to help until now so it is time to change that and I hope other lurkers will take this opportunity to join the fight against the slow phasing out of 3D video games.

+++ wish some more would do the same *thumpsup*

[for3donly]

fantastic! I thought this project might take a few hundred man hours to complete but it seems you can have it done in just a fraction of that, you makes it looks easy which hopefully will encourage someone to take on dx12 wrapper challenge.

[Marin3D]

Insane project! mucho respect, the amount of accumulated knowledge to do a project like this is insane.

@for3donly, calculated in man hours, this will not be a fraction of a few hundred, don't underestimate all the work that has to be done still (and is already done).

I really hope that having the Vulcan renderer will keep 3D vision alive in the coming years. I still consider 3D vision on a 27" 1440 resolution superior for 3D over a current gen HMD.

[Marin3D]

double submit

[helifax]

fantastic! I thought this project might take a few hundred man hours to complete but it seems you can have it done in just a fraction of that, you makes it looks easy which hopefully will encourage someone to take on dx12 wrapper challenge.

This is definitely not a "walk in the park". It definitely requires a lot of time to get it to the place I want/needs to be.
For anyone interested to even attempt something like this, you will definitely need quite a few skills under your belt:
- Very strong C/C++ programming experience, (and programming in a multi-threaded environment - in Vulkan, DX12). Gone are the days of where the whole rendering pipeline was on one core (like DX9/11 and OpenGL).
- A lot of Windows/Linux API knowledge and how things work under the hood in general.
- A strong knowledge of graphics workflow, programming, tools and debugging. (You don't need to be an expert in writing shaders, or a rig animator, but you definitely need to be familiar with how graphics engines work and the workflows).
- Reverse Engineering of software in general is always very helpful (You need to train your mind in a certain way.) & in-depth debugging.
- Reading the documentation of an API and understanding (as much as possible) all the details (the devil is in the tiniest detail).
- Experimentation, experimentation, experimentation.
- A strong will (Things will never ever work out on your first attempt. Sometimes you'll be stuck on a problem for days ^_^).
- Previous experience in working with graphics at the lowest level (basically, all the "to-do" and "do not do" that you learnt in the past ^_^).

Regarding Vk3DVision, there are still a lot of things to do, but I am trying to tackle one problem at a time and also code the interaction between the UI and code and the debugging functionality as early as possible. This not only helps me test out all the code & functionality that I wrote, but it also acts as a validation for me to see if I actually understood all the aspects of the problem that I am trying to solve
Also, the code is always cleaner if the UI was thought of in the beginning and it's not something that was "slapped" on top as an "aftermarket" treatment

Cheers,
Helifax

[helifax]

In the last days/week I've done quite a few things:
- Finally managed to fix a graphics corruption/driver crash/pc hard lock related to the Description Sets & Layouts injection & modification. Hopefully, I'll never see this ever again as it was quite tricky to find.

- Worked on the Ability to Change the Shader Source (while the game is running), Recompile & Swap:
This sounds easy, but actually it's the other way around. If you are familiar with Vulkan then you know why. If not, here is the short-version:
In Vulkan, you don't bind individual Shaders like you do in DX11 (No PSSetShader()). Instead you bind a Pipeline. This pipeline contains all your shaders, rasterization state, viewports, vertex primitive topology etc. Basically, every single aspect of the pipeline state. You bind the pipeline, the description sets, etc, update your Uniform, Samplers, Push Constants etc, and DRAW! This is part of a Command Buffer that is then submitted. Then your repeat for your next draw call
In order, to swap the shaders, I basically have to regenerate the WHOLE pipeline of the shader that I want to swap. Easier said than done (since the Pipeline creation structure is full of pointers, which get invalidated after the initial creation. So... Deep Copy time baby )

I have now have full ability to change shaders & recreate the corresponding pipeline (This will allow me to quickly find a broken shader and try to fix with from within the game! No alt-tabbing, just editing, pressing a UI button and see the results )

- A bit of work on showing the original version of the shader running. (I haven't added the ability to SKIP a shader rendering yet. I am investigating ways of doing this).


And lastly, I actually recorded a few Minutes from Detroit: Become Human rendering in Stereoscopic 3D:

https://www.youtube.com/watch?v=EYQHw9aMAsI


(Grab the video and watch it with your stereo3D player).
Sure, there are rendering issues, broken shadows, etc. I simply didn't touch any shader yet. This is all done using the Automatic Stereo Injection that I have implemented (like in my OGL3DVision wrapper).
Ignore the loading stuttering for now, this was running a debug (no optimisations) build with the debugger attached.

But, you definitely see the depth and 3D Environment!

Hope you like it!

Cheers,
Helifax

[for3donly]

Hey guys I definitely thought this was a "pro" level project that requires a wide and deep skills set, don't get me wrong.

@Helifax you are taking on a huge challenge and I have only admiration and respect for your passion and talents. You were super brave stepping up and assume such a huge project while having to work your day job and everything else. That's why i didn't hesitate to show my support the moment i came across this thread. You are in a sense the fearless hero that we play in games. seriously. at a time when nvidia abandonned, vr alternatives are there, 3d vision is clearly on the decline it's naturally that pp with talents would feel dissappointed and not engaged but you didn't. you take on the biggest task and conquer it, that takes a lot of guts and passsion, you are our community hero.

[helifax]

Hey guys I definitely thought this was a "pro" level project that requires a wide and deep skills set, don't get me wrong.

@Helifax you are taking on a huge challenge and I have only admiration and respect for your passion and talents. You were super brave stepping up and assume such a huge project while having to work your day job and everything else. That's why i didn't hesitate to show my support the moment i came across this thread. You are in a sense the fearless hero that we play in games. seriously. at a time when nvidia abandonned, vr alternatives are there, 3d vision is clearly on the decline it's naturally that pp with talents would feel dissappointed and not engaged but you didn't. you take on the biggest task and conquer it, that takes a lot of guts and passsion, you are our community hero.

No, I totally understood you! I wasn't trying to tell you otherwise.
I wrote my reply to better reflect exactly what you said and why you are right to think that a project like this could take up to hundreds of hours of work
I wanted to help and to further detail the skills and knowledge that one might need to have if anyone wants to tackle something like this (You don't have to be skilled in all the above, if you are willing to learn and have a lot of patience )

As a tiny side-note:
In the end, a piece of software gets to EOL the moment it stops receiving updates and people stop working on it. I've never eve seen a "complete" software (there are always bugs to fix, new features to add, or to improve different aspects. But, at some point, when everything is stable enough, you can make a "Release").


Cheers,
Helifax

[Chtiblue]

In the last days/week I've done quite a few things:
- Finally managed to fix a graphics corruption/driver crash/pc hard lock related to the Description Sets & Layouts injection & modification. Hopefully, I'll never see this ever again as it was quite tricky to find.

- Worked on the Ability to Change the Shader Source (while the game is running), Recompile & Swap:


I have now have full ability to change shaders & recreate the corresponding pipeline (This will allow me to quickly find a broken shader and try to fix with from within the game! No alt-tabbing, just editing, pressing a UI button and see the results )

- A bit of work on showing the original version of the shader running. (I haven't added the ability to SKIP a shader rendering yet. I am investigating ways of doing this).


And lastly, I actually recorded a few Minutes from Detroit: Become Human rendering in Stereoscopic 3D:

https://www.youtube.com/watch?v=EYQHw9aMAsI


(Grab the video and watch it with your stereo3D player).


But, you definitely see the depth and 3D Environment!

Hope you like it!

Cheers,
Helifax

Awesome 8)
Thanks for the daily report

[helifax]

I give you the FIRST semi-PERFECT STEREOSCOPIC 3D rendered image from Detroit: Become Human:


(Sure, the UI is not "in-depth". I don't really care about "UI" at the moment - Didn't we become "picky" over the years ^_^ ).
(There is ONE thing that is wrong in this image though Can you spot it )

HOWEVER, there is catch (like always )
It's actually simple to explain: The same shader can be used multiple times in different pipelines (contexts). Just plain hashing a shader, exporting it and modifying it, WILL NOT WORK!.
So... here is the Challenge:: - What makes a pipeline UNIQUE? (I need to find a way to generate UNIQUE HASHES for shaders based on the pipeline they are being USED IN!.

So... How, where, what (Brain storming event "on the schedule" )

In the meantime, hope you <3 the image above

Cheers,
Helifax

[for3donly]

Very nice. 3d effect lools really good. something wrong you mean her straps got some odd artifacts ?

[helifax]

Very nice. 3d effect lools really good. something wrong you mean her straps got some odd artifacts ?

Bingo!

I didn't fix those wrong shadows there! Very well spoted!

Now, is the next part... making this "deterministic" so I can replicate the same thing when loading the "changed" shaders from disk!
("Here comes the Unique shaders" part ^_^).

I owe you a beer! For the keen eye

Cheers,
Helifax

PS: Currently I am running with a fixed Depth of 100% (in Nvidia Control Panel for Stereo3D) and a fixed convergence
I don't have the code (yet) to change the converge on the fly;) (I simply didn't added it, as I have bigger "fishes" to fry I'll add that at some point! Rather sooner than later as it seems I'll need it )

[for3donly]

you remind me of these pubs in germany they laughed when i asked for ice for my beer. Yeah i noticed the extreme depth in the video as well. 100%! no wonder.