Yes, but many games support thumbsticks on XBox360 controller - don't they?

This is what I am currently working on with FriiSpace. Translating real world motion into emulated 360 thumbsticks in SkyRim.

Awsome work Pierreye !
An impressive HMD Setup you got there !

Thats incredible ! Must be totally immersive for you to play FPS Games/Simulations etc.
Hopefully we can try that too with the ST1080 soon.

By the way, as far as i know the Infantry Simulation ARMA 2 and ARMA 3 support dual input ( headtracking and mouse input )
So you can look around with your Headtracking device and aim with your Mouse.

Watch this Video please ( i know its old , may be you already saw it, but its worth a try )
http://www.youtube.com/watch?v=9wXx3vMy_AQ

Getting the camera matrix out of directX is not simple, there is no real concept of camera matrix, it's just an input to a shader, and worse several can be used in a single frames worth of rendering. I'm actually not sure how NVidia deals with this in the 3D Drivers, best guess is they identify the matrix based on the contents.
It would probably be doable on a game by game basis, by doing a shader capture (like Helix' shader mod for 3DVision) and figuring out which input it is for a specific commonly used game shader.
Doing it automatically the only way I can see it looking at the contents of the matrix, and identifying the usual camera matrix pattern, but your back to trying to determine if it's the camera for the main view or an offscreen surface, NVidia doesn't have to worry about this because they use other mechanisms to determine if they need two cameras or not.

Everything falls out of the camera matrix if you have it.

Its been an extremely long time since I did any Direct3D stuff, but surely this is just an intercept to SetTransform or the modern equivalent when its doing the view transform. Since you pump that all your transforms, you should be able to intercept the one you want.

It can't be that hard to do, or 3D drivers such as DDD and Nvidia wouldn't be able to do it so well?

To be honest, I am still somewhat of a noob when it comes to DirectX. Much of the 3D work I've done up until this point has been with higher-level engines (like OGRE) which hide much of the nitty-gritty details. From what I get so far, there are certain functions that will always be called around the place where the rendering happens. For example "BeginScene()" "EndScene()" and "SetTransform()" in particular. So I think I am on the right track, but its probably not as easy as it looks. Though surely if people like iz3D or DDD can have a somewhat generic solution it must not be impossible.

It seems like if you used some type of hooking or DLL replacement, you could intercept the SetTransform call. The first argument is an enum and you could just check for D3DTS_VIEW (0x2).

ARMA 2 is using TrackIR for headtracking. Not sure if GlovePIE TrackIR support ARMA 2 as few years back, TrackIR encrypt their API and interface so most third party solution doesn't work.

*Commercial begin* It supports FreeTrack too, which FreePIE supports *Commercial ends*

The problem is in DX9+ there is no concept of a model/view transform, it's just a matrix input to a shader, there is no context. It's not like the old fixed function stuff where you set the transforms explicitly.
You'd have to look for things that look like camera matrices being passed to shaders, that's not particularly hard, they have a very specific pattern of entries, but you have no idea if it's just the camera transform (or if some of all of the model transform is included), or if it's the camera transform for a shadow map rather than for the primary render target.
Though I guess you can track which render target is selected for the latter.
My guess is you could get it to work for a lot of games pretty quickly, but you'll have outliers that are harder to get working.

Sorry I missed this, I think the reason NVidia and DDD can do it is the modification doesn't require they isolate the camera transform explicitly, they just have to produce the same result as adding an additional translation into the final matrix.

@ERP: Sure once the matrices all get multiplied together in the render-pipeline it would be difficult to disentangle them reliably. But what if you trapped the view matrix way back up at the initial input function? (ie. SetTransform(D3DTS_VIEW, ...) ) I guess some games might pre-compute the combined matrix and input it all at once, but surely lots of games input the view matrix independently using this method. What am I missing here?

PS. My graphics is rusty and mostly OpenGL based.

I don't know enough to comment here, my DX experience ended with V6 I think.

At a glance tho Brantlew's concept seems workable...

Here's another VR demo for Mass Effect 3. This clip is longer around 12 mins of unedited gameplay to prove that the whole VR setup is playable.

Cool video!

Is it odd playing in 3rd person view like that?

Sick man! Very nice.

It's playable once you adapt. The view is much better than DeadSpace 2 as when you turn, the character will turn together so the back is always facing you. DeadSpace 2 when you turn, your view actually shift to the front of the character. I can aim and shoot better than using a mouse/keyboard combo as no longer need to lift the mouse when doing a quick turn and fine aiming feel much more natural.

Nice!
I tried Mass effect 3, but my eyes got tired of all the halo effects around the characters, to bad ME3 is not capable of correct stereo 3d :/

Try out Helix DirectX 9 3D Vision fix.
http://helixmod.wikispot.org/gamelist

Out of topic, ancjob try to spam my youtube video, damn arsehole.

Will do, thanks! Is ME3 a DX9 game? Thought most games where 10 - 11 these days

about ancjob, unemployed people with too much free time are dangerousness...

You will be surprise most game still using DirectX 9. ME3 and Skyrim both are directx 9.

Its because of all the f*ing console ports :/

Try out Battlefield 3 and the 3D is good. With the config.ini tweaking the depth and convergence, the 3D had lot of pop and depth. The headtracking in the Jet fighter is just damn realistic. Only hate the dirty lens effect.

Yeah, its really nice, really love how you cna make bf3 take away the stereo 3d when aiming down the scope... I hate it that they didnt implement proper freelook.. That you cant stear the helcopter while freelooking...

btw, try Alan Wake, its really nice after the latest patch

Can you guys suggest best 3d games that can also incorporate mouse view tracking via the wiiremote on the HMZ.

Obviously the FPS type games - COD, Skyrim, MAss Effect, etc that I see in with Pierreye's setup however wondering if there are some good racing, sims, etc that can also use the HMZ/wiimote setup. Thanks!

FSX and Black Shark are two of my faves if you are into sims...

Ah yes - I do have FSX - I shoudl reinstall that for the HMZ

I do remember that you had to hold down the space bar to activate mouse look... is there away around this? I do recall when I used the Vuzix glasses the driver automatically set this...

Hmm, not sure, I vaguely remember coming across that issue, but theres an option or something like that that lets you turn it on permanently. Thats what I have done. Can't remember what, but if you google it you'll find the answer (probably modifying a .ini file).

Going to try recently release Wing Commander Saga developed by fans using Open Freespace engine. Too bad it is in OpenGL so no 3D support. FS2Open should support headtracking through TrackIR API. I love Wing Commander and FreeSpace series.

Grid race driver and latest Dirt both have TrackIR support

Wanted to let you guys know that the motion tracking components in the Razer Hydra can be fit into a very, very small tracking cube, about 10x15x20mm perhaps? I will take pictures tomorrow, I have a cable coming out of the right hand controller (Which now has no tracking) to a small box containing the tracking components. Mounts to an HMD perfectly!

Here is a picture (Borrowed from WiredEarp) that I put lines on showing where I made cuts:


All three coils share a common ground, and each also has a positive line, for a total of just 4 wires that need to be relocated. Soldering to the tracker side is very easy, but be careful on the main circuit board end! You have to scrape the enamel off the traces and solder directly to that, and the traces are very thin and easy to lift off the board. I accidentally did that with one of my connections, but I was able to wire directly to where the trace led on the board, phew! My advice would be to hot-glue the wires to the back of the board for strain relief, and then solder the connections.

Tracking still works flawlessly, even extended with 4 feet of wire! It might just be the placebo effect, but it seems like it might work even better than before.

Hey PalmerTech, What did you do about the small 'daughterboard' that contains the buttons, triggers etc? Just leave that connected?
Be extremely careful with the cable that connects that to the main board, it has failed in mine now with all the moving around when testing, and I haven't been able to figure out a replacement yet (way too tiny for me to solder I think). If you are leaving it in the existing Hydra shell that is probably a good idea to protect it from movement.

Glad to hear you can reduce the size further doing that! It looked possible (as did merely bending the board so it reads 'flatter', as mine had to be mounted on a slight upwards angle to read 'flat') but I was afraid it would be timing the signals from those coils and it could throw things off. Great to hear it doesn't. I'll try your technique here when I get my Hydra going again. It already makes an awesome head tracker, but reducing it like you have mentioned will mean it will fit on my P5 glove much better.

Only issue with using the Hydra is you need the buttons etc to initialize it. I REALLY wish Sixense would just release a version of the Hydra tracker designed for tracking specifically.

I left all the other components in the controller. I can either build it into my motion tracked gun to use as extra buttons and analog stick, or just put it in the backpack. I think this is the best solution, since it leaves the controller fully usable aside from motion tracking. Next, I want to put some kind of connector on the module. That way, I can either use it for headtracking with an extension cable, or plug it right into the end of the Hydra for "normal" use.

Great idea. Should be easy to use any sort of micro connector to do that.

Where would you put the module bit when you try to use it like a standard Hydra again? Would you just cut out a section of the Hydra controller handle so you can stick it in when you need it, or mount it some other way?

@Pierreye Have you heard anything from this guy who claims to have got the Move to work fully work without the wiimote? He was also suppose to post an executable for it.:

Not yet. He previously mention that MoveOnPC is conflicting with MotionInJoy driver so I'm not sure how he hook PS Move and PS Navigator to the PC. I'll check with him.

Ok great. Looking forward to what you find out.

Managed to tie the wiimote to the leg as detection for prone/jump and it works. I'll shoot a battlefield 3 demo soon with headtracking, gun tracking and leg motion detection. It's also possible to disable movement if you don't move your leg.

Did you ever get to make this demo? Get the setup to work as he did without the wiimote?

Hello guys!!!! My name is Pedro, I´m from Brazil!!! Work in game industry as level designer and 3d modeler!!!
I´ve been following this topic for some time and i have to say its great!!!!

I dream about this for at least 8 years!!!! But here in Brazil its so dificult to get an HMD (too much expensive!!! eaven the simple models!!!) I started to build my on setup months ago. I´m not really gonna use this for games but mostly for realtime archviz (games are a bonnus!!!! ) I´m making my house in UDK and I want to walk in it before I start building.

This is what I did already.

Manage to get wiimote + motion plus and ps3 navigation working great with pierre scripts in Glovepie + MotionnJoy and 2 bluetooth adapters.
Vuzix Wrap 920VR (just got mine and I´m really enjoying it) using mouse emulation app from cybereality.

The problem is that can't get then working together!!!! Carl is still banned from Vuzix!!!! Does it work with older version of glovepie???? 0.3???

@Cybereality how did you get it working????? You wrote other app for this???? Can you please upload it????

And the 3D effects work with nvidia stereo drivers???? I only get the anaglyf mode working the others uses nvidia glasses right?

Thank you all!!!!! Keep up the good work!!! If you guys need anything that I can do i´ll be happy to help.