Console games

[2EyeGuy]

I thought I'd make a start on getting console games supported. Starting with every Wii and Gamecube game. So far I've got a modified Dolphin emulator that can roll the camera in response to head tracking. The other axes are easy too, but don't look as good. I want to fake the input to make the game turn properly instead for games with camera controls. And I need to get it to render side-by-side in 3D and the correct aspect ratio, but that shouldn't be too hard. And I need to get the motion plus gyros to pretend to be a sensor bar. And there are some other things that need doing to tighten up the experience. I might try doing Project 64 too. Then we will have the "Nintendo On" for real.

[cybereality]

Interesting. Love to see this working.

[2EyeGuy]

No 3D yet, and these screenshots don't have the right vertical resolution since my laptop screen only goes up to 1366x768. But there is extra vertical FOV, and the correct aspect ratio (mostly), and the HUDs (mostly) are in their original dimensions in the middle of the screen. And I forgot to take a screenshot of it, but roll is working. Some of these games have some issues though.



Attack of the Movies 3D (ironically still in 2D):

Dead Space Extraction:

GameCube games:



These are on the PC, in a modified version of Dolphin. But we think we might be able to get something working on a real Wii too, if the Rift can accept at least one plug/signal/resolution that the Wii is capable of outputting.

If Sony and Microsoft don't want to support the Rift, they can just miss out.

[brantlew]

Very cool!

[Oneironaut]

F-Zero GX in VR... *drool*

I'll be following this closely!

[FingerFlinger]

Nice work! Project64 support would be cool as well. I'd love to see a hack that lets the user stroll around using map data from Ocarina of Time or Super Mario 64, Goldeneye...

[Fredz]

Nice job indeed !

[Omarzuqo]

You're CarlKenner in the dolphin forums, right? I've been lurking your thread.

http://forums.dolphin-emulator.com/show ... ?tid=25483

I'm sorry I have nothing relevant to tell you, but I have to say, I'm really impressed and grateful for your progress.

The first thing I thought when I heard about hawken was the Rogue Squadron games for the Game cube, those have a cockpit view mode with free look, just like hawken, too bad those games don't even boot on Dolphin.

One more thing, If you get Metroid Prime working I will worship you eternally.

[brantlew]

You're CarlKenner in the dolphin forums, right? I've been lurking your thread.

Carl Kenner? Like the GlovePIE Carl Kenner?

[cybereality]

Wow dude! Nice work.

[2EyeGuy]

Dolphin still doesn't support Metroid Prime Trilogy, unless you modify the iso file using Wii Scrubber to replace the main.dol file with one of the other 3 dol files AFTER you have played through the menu once and selected a character. Dolphin is really not as good as any of the Wii loaders in terms of loading alternate dol files. Also I'm getting about 5 FPS in Metroid Prime 1 at the moment (in debug mode though).

The interesting thing about doing Metroid Prime is that she is wearing a helmet the whole time. It's going to cut off some of her vertical FOV. And as far as I can tell, each of her eyes can only see part of the HUD. I'm not sure how well that will work in real life, but it will be fun to try out.

Metroid Prime uses a bunch of different projection matrices, so hopefully I'll be able to isolate which one it's using for the helmet and use a different matrix for it, so it renders like a real helmet, and make it rotate with your head rather than the world. (She probably can't roll her head much though, and shouldn't be able to see anything when she turns her head, with those shoulders.) Hopefully the cross-hair and gun aren't rendered in the same matrix as the helmet.

[Omarzuqo]

Dolphin still doesn't support Metroid Prime Trilogy, unless you modify the iso file using Wii Scrubber to replace the main.dol file with one of the other 3 dol files AFTER you have played through the menu once and selected a character. Dolphin is really not as good as any of the Wii loaders in terms of loading alternate dol files. Also I'm getting about 5 FPS in Metroid Prime 1 at the moment (in debug mode though).

The interesting thing about doing Metroid Prime is that she is wearing a helmet the whole time. It's going to cut off some of her vertical FOV. And as far as I can tell, each of her eyes can only see part of the HUD. I'm not sure how well that will work in real life, but it will be fun to try out.

Metroid Prime uses a bunch of different projection matrices, so hopefully I'll be able to isolate which one it's using for the helmet and use a different matrix for it, so it renders like a real helmet, and make it rotate with your head rather than the world. (She probably can't roll her head much though, and shouldn't be able to see anything when she turns her head, with those shoulders.) Hopefully the cross-hair and gun aren't rendered in the same matrix as the helmet.

Wow, I didn't think you would even consider my shameless brat-ish request.
You're my new favorite person.

[Bretspot]

Dude Metroid Prime VR would be mind blowing!!!!11
I don't think it would ever be perfect for VR though without a complete redesign. Who knows, maybe Nintendo Wii-VR will come out in 2016 and they will re-release the game and re-blow our minds again.

[biolizard89]

Hey everyone,

I'm the developer of GeckoTunnel, a software project which, among other things, aims to implement head tracking for GameCube and Wii games (on both actual GameCube/Wii hardware as well as Dolphin). You may have noticed me in the Dolphin forum thread. I don't yet have head tracking working, but I do have stereo 3D output (field-sequential 480i and frame-sequential 480p) working in F-Zero GX (tested on a Wii).

Those of you who have F-Zero GX (tested with the USA version, but will probably work with the PAL and JPN versions too) and a homebrew-enabled Wii (it should work on a GameCube too) can find the code at http://geckocodes.org/?c=GFZA01 (it's a GeckoOS code).

It's a proof-of-concept; I plan to refine it a lot... but hopefully some people here will like it.

[2EyeGuy]

That's amazing!
Unfortunately, my evil Samsung 3D TV won't support 3D mode for that input, or any other signal that the Wii will put out with the default yellow/red/white AV cables, so I won't be able to test it, but I'm very impressed. I don't know if there is a cable available that a Samsung 3D TV will accept and a Wii can output.
Anyway, I'm house-sitting at the moment, and I don't have my Wii here, then I'm going to Europe in a few days. So I can't do any testing for a few weeks.

We don't know what kind of inputs the Oculus Rift will accept, probably not something the Wii can output directly, but it needs to be in Side By Side format. I imagine that's harder. Maybe it can be done just with viewports in some games, but other games will read the screen back into a buffer then write that buffer to the screen again, which would make some elements have 4 images side by side instead of 2.

[biolizard89]

That's amazing!
Unfortunately, my evil Samsung 3D TV won't support 3D mode for that input, or any other signal that the Wii will put out with the default yellow/red/white AV cables, so I won't be able to test it, but I'm very impressed. I don't know if there is a cable available that a Samsung 3D TV will accept and a Wii can output.
Anyway, I'm house-sitting at the moment, and I don't have my Wii here, then I'm going to Europe in a few days. So I can't do any testing for a few weeks.

We don't know what kind of inputs the Oculus Rift will accept, probably not something the Wii can output directly, but it needs to be in Side By Side format. I imagine that's harder. Maybe it can be done just with viewports in some games, but other games will read the screen back into a buffer then write that buffer to the screen again, which would make some elements have 4 images side by side instead of 2.

Yeah, side-by-side would be much harder. Right now I'm just poking the camera XYZ coordinates right after the game calculates them... side-by-side would require messing with a lot of other stuff.

That said, I'm using a $30 capture card to feed the field-sequential output through Stereoscopic Player and into my Nvidia 3D Vision display. I honestly think a capture card is the least-hassle way to use a Wii in 3D with a display that doesn't support field-sequential. I don't notice any lag in my setup, although I admit I'm not trained to recognize tiny amounts. (The choice of capture card makes a difference... some have no lag, while others have a lot. I had to ask around to find a good one.)

[troffmo5]

You did undoubtedly a good work
Of course most of us has still no Oculus, but is this the best way to see a Wii game on it? Your screenshot shows the game running fullscreen, that means that all the pixels are used (excluding the corners, due to the distortion), but are all the pixels visible on the Oculus?
The Rift is made to be immersive and for this reason it has a big FOV, but the GC/Wii games are made to be viewed on flat screens and i doubt that stretching them to the complete visible area makes them more immersive.
Perhaps for such games it is better to use a virtual screen to put in front of the viewer, if possible, with stereo effect. Technically you had only to reduce the viewing area.
Tell me if i am wrong, because i can't test it on the Oculus yet

[biolizard89]

You did undoubtedly a good work
Of course most of us has still no Oculus, but is this the best way to see a Wii game on it? Your screenshot shows the game running fullscreen, that means that all the pixels are used (excluding the corners, due to the distortion), but are all the pixels visible on the Oculus?
The Rift is made to be immersive and for this reason it has a big FOV, but the GC/Wii games are made to be viewed on flat screens and i doubt that stretching them to the complete visible area makes them more immersive.
Perhaps for such games it is better to use a virtual screen to put in front of the viewer, if possible, with stereo effect. Technically you had only to reduce the viewing area.
Tell me if i am wrong, because i can't test it on the Oculus yet

I'm not sure I completely understand your post; are you implying that the FOV of GC/Wii games will not work well on the Rift? It's not hard to adjust the FOV which the games render at; emulators like Dolphin and console tools like Swiss and DML have done this for years (to force widescreen rendering). Some games work better at nonstandard FOV than others, due to geometry culling... I don't think anyone's fully investigated how easy/hard that is to fix. (I plan to investigate that when I have time; I think 2EyeGuy is planning to play around with that too.)

Obviously, games not designed for VR will have some bugs, some of which can be worked around with hacks, while others may not be easily fixable. I don't think there's any difference between PC games and GC/Wii games in this respect, other than the fact that the Wii's 480p60 output is lower quality than what most PC games output. (Then again, 60fps on a Wii can be more than the framerates some PC games give.)

Have I misunderstood your post?

[troffmo5]

Thanks, i was ignoring to possibilty of tweaking the FOV on some emulators (Dolphin included). In that case limiting the HMD FOV makes no sense.
I found a post with a list of emulators supporting FOV customization:
http://www.wsgf.org/forum/12158/emulato ... ov-options

and here a list of games supporting multiple FOV with instructions:
http://www.wsgf.org/mgl
useful for injection drivers.

[biolizard89]

That's amazing!
Unfortunately, my evil Samsung 3D TV won't support 3D mode for that input, or any other signal that the Wii will put out with the default yellow/red/white AV cables, so I won't be able to test it, but I'm very impressed. I don't know if there is a cable available that a Samsung 3D TV will accept and a Wii can output.
Anyway, I'm house-sitting at the moment, and I don't have my Wii here, then I'm going to Europe in a few days. So I can't do any testing for a few weeks.

We don't know what kind of inputs the Oculus Rift will accept, probably not something the Wii can output directly, but it needs to be in Side By Side format. I imagine that's harder. Maybe it can be done just with viewports in some games, but other games will read the screen back into a buffer then write that buffer to the screen again, which would make some elements have 4 images side by side instead of 2.

Yeah, side-by-side would be much harder. Right now I'm just poking the camera XYZ coordinates right after the game calculates them... side-by-side would require messing with a lot of other stuff.

That said, I'm using a $30 capture card to feed the field-sequential output through Stereoscopic Player and into my Nvidia 3D Vision display. I honestly think a capture card is the least-hassle way to use a Wii in 3D with a display that doesn't support field-sequential. I don't notice any lag in my setup, although I admit I'm not trained to recognize tiny amounts. (The choice of capture card makes a difference... some have no lag, while others have a lot. I had to ask around to find a good one.)

As an aside, there are some games where it should be possible to use the 2-player split-screen mode to simulate side-by-side or top-bottom stereoscopic 3D. This wouldn't work for single player mode though. Two Wiis/GameCubes/Dolphins would be used (one for each player); GeckoTunnel's LAN mode would sync the game state between consoles, and the remote player's camera on each console would be redirected to the position of the local player. This would produce increased compatibility for some displays which don't support field/frame-sequential, and would double the framerate, but would of course halve the (already low) resolution. By itself, this still wouldn't make it work with the Rift directly since there's still a warping transformation which is necessary... but this would drastically improve the experience with 30fps games, seeing as 15fps per eye would probably make for a lot of motion sickness.

This is on my long list of things to play around with.