Head Tracking/Gun controller help and suggestions required!!

[pierreye]

I don't think that is necessary as you can get the absolute pitch from wiimote.pitch which show you the degrees but the problem with wiimote.pitch is a bit jumpy to use for smooth aiming. A quick scroll up and back to center might not work in sniping mode which is quite annoying. I do experiment with auto center which detect if the pitch is within 1 degrees from absolute horizontal level every 10 secs, it will auto center in COD series. I found out it kind of screw up my aiming that it auto-center when I'm about to shoot the enemy especially in sniping mode.

Another update on the roll compensation that I try out earlier in COD:MW3 and it work perfectly. The drift had been minimized and only to recenter occasionally. You can now hold your rifle 90 degree and still the X,Y movement is working correctly. This setup would be great for Dead Space 2 where you can turn 90 degree to fire in alternative mode while having the correct movement.

[CyberVillain]

IN arma2 you can use TrackIR and freetrack at the same time, so if you bind Freetrack to headtracking and TracKIr to movement you could have absolute movement both for looking around (headtracking) and when moving body (Moving your weapon)

Youll need two wiimotes offcourse and you need to calibrate the headtracking one to the other so that where the weapon is pointed is center for the headtracking yaw.. Would be reaaaaaly cool

edit: sorry, I just assumed that you could have absolute movement since the game has absolute movement for the head. But i tried to assign the movement to my freetrack, and its relative :/ Really stupid of them

[pierreye]

Here's the updated script with roll compensation.

[Syntax]

Seems like we are reinventing the wheel in here

My setup i have posted before used a hmd, with a gyration air mouse mounted on top and a wii zapper gun consisting of a wii remote, motion+ and nunchuck mounted in a cheap plastic gun. Basically it is dual mouse emulation with bound keys for movement and other options. You can hold a button on teh wiimote to use teh tracking or not so you can easily switch between head or gun tracking or both, it works quite well. Cant recommend gyration air mouse any more highly, these things are great.

(I think) this the wii script i used, the mouse tracking is jittery but fun enough. Note i always bind 'e' to jump in my fps and space to duck, makes sense right?

Quote:
Key.W = Wiimote1.Nunchuk.up
Key.a = Wiimote1.Nunchuk.left
Key.s = Wiimote1.Nunchuk.down
Key.d = Wiimote1.Nunchuk.right
Key.E = Wiimote1.Nunchuk.ZButton
Mouse.LeftButton = Wiimote1.Nunchuk.CButton

if starting then
// define deadzone, calibration and sensitivity
var.calibrationGyro = [-1.5, -0.5] // [Yaw, Pitch]
var.deadzoneGyro = [7, 7] // [Yaw, Pitch]
var.sensitivityGyro = [0.55, 0.65] // [Yaw, Pitch]; lower values result in slower movements, negative values will invert the corresponding mouse axis
endif

// hold B to move mouse with WiiMote
if (WiiMote.B) then
var.Gyro = [0, 0]
if (Abs(WiiMote.MotionPlus.YawSpeed + var.calibrationGyro[1]) > var.deadzoneGyro[1]) then var.Gyro[1] = RemoveUnits(WiiMote.MotionPlus.YawSpeed + var.calibrationGyro[1]) - (Sign(WiiMote.MotionPlus.YawSpeed) * var.deadzoneGyro[1])
if (Abs(WiiMote.MotionPlus.PitchSpeed + var.calibrationGyro[2]) > var.deadzoneGyro[2]) then var.Gyro[2] = RemoveUnits(WiiMote.MotionPlus.PitchSpeed + var.calibrationGyro[2]) - (Sign(WiiMote.MotionPlus.PitchSpeed) * var.deadzoneGyro[2])
Mouse.DirectInputX = Mouse.DirectInputX + (var.Gyro[1] * var.sensitivityGyro[1])
Mouse.DirectInputY = Mouse.DirectInputY + (-1 * var.Gyro[2] * var.sensitivityGyro[2])
endif
if wiimote.b = false then var.1 = 2
if wiimote.b = true then var.1 = 10

mouse.LeftButton = wiimote.A
mouse.RightButton = wiimote.Plus

if wiimote.down = true then mouse.directinputy = mouse.directinputy + var.1
if wiimote.up = true then mouse.directinputy = mouse.directinputy - var.1
if wiimote.left = true then mouse.directinputx = mouse.directinputx - var.1
if wiimote.right = true then mouse.directinputx = mouse.directinputx + var.1


The first thing i realised once i got this setup is that i needs real movement tracking

Great work Okta, but I ve a problem with the nunchuck.
When I start the script with no nunchuck plugged in the tracking(gyros) is working fine.
But when I plug the nunchcuk in and start the script I can only use the nunchuck means the tracking is mysteriously disabled. Also have tried to add a second wiimote but there was the same probelm. So How can I track and use the nunchuck at the same time and is it possible to only track with the first wiimote and use the second as controller?

Plz help

P.s: I finally got my hands on my very own HMZ-T1 and its so amazing once you make it comfortble and set the picture settings right.

[pierreye]

My previous script using Nunchuk with MotionPlus had the same issue.

What you need to do is connect the Wimote + Motion Plus without Nunchuk first. Click RUN in GlovePIE and connect the Wiimote by pressing 1 + 2. Once connected and calibrated, stop the script and plug in the Nunchuk. Then run the script again and it should be working fine. Again, I suggest getting the PS3 Navigator as it come with more button and wireless. You can press down the joystick as "Shift" for either sprint or steady sniping mode.

[CyberVillain]

My previous script using Nunchuk with MotionPlus had the same issue.

What you need to do is connect the Wimote + Motion Plus without Nunchuk first. Click RUN in GlovePIE and connect the Wiimote by pressing 1 + 2. Once connected and calibrated, stop the script and plug in the Nunchuk. Then run the script again and it should be working fine. Again, I suggest getting the PS3 Navigator as it come with more button and wireless. You can press down the joystick as "Shift" for either sprint or steady sniping mode.

Cant you detect when you hold your breath and steady the scope?

[Syntax]

What you need to do is connect the Wimote + Motion Plus without Nunchuk first. Click RUN in GlovePIE and connect the Wiimote by pressing 1 + 2

my wiimotes are already connected via my toshiba bluetooth driver software, so I dont know what you mean by saying "press 1+2 to connect"
They are allready connected.
But when the nunchuck is pluged in the gyros are disabled... WHY???

[pierreye]

My previous script using Nunchuk with MotionPlus had the same issue.

What you need to do is connect the Wimote + Motion Plus without Nunchuk first. Click RUN in GlovePIE and connect the Wiimote by pressing 1 + 2. Once connected and calibrated, stop the script and plug in the Nunchuk. Then run the script again and it should be working fine. Again, I suggest getting the PS3 Navigator as it come with more button and wireless. You can press down the joystick as "Shift" for either sprint or steady sniping mode.

Cant you detect when you hold your breath and steady the scope?

That's pretty hardcore for VR

[pierreye]

What you need to do is connect the Wimote + Motion Plus without Nunchuk first. Click RUN in GlovePIE and connect the Wiimote by pressing 1 + 2

my wiimotes are already connected via my toshiba bluetooth driver software, so I dont know what you mean by saying "press 1+2 to connect"
They are allready connected.
But when the nunchuck is pluged in the gyros are disabled... WHY???

I'm using standard MS Bluetooth stack and always have to press 1 + 2 on Wiimote for first connection to GlovePIE. In this case, just connect first without Nunchuk, then stop GlovePIE and plug in Nunchuk. Run GlovePIE the second time and it should work.

[mayaman]

Your video on YouTube is unreal. I'm hoping to emulate it. Please upload more videos and hopefully some tutorial videos for the more "green" among us.

[Syntax]

thx guys for all your suggestions.
Finally I got it to work. When your Wiimotes are already connected, I had to start the script first without motion plus but when the script starts to calibrate the wiimote I plugged in the motion plus + nunchuck whlie the script is running and there you go

[Max3d]

Hi guys,
Completely new here. I was looking for the guy who posted the amazing HMZ-T1 wireless VR FPS gaming video on youtube. Uploaded by rjyap1 on Dec 27, 2011
I would like to hear how he build this setup or something similar for a blog I maintain. It´s focused on the European Tech scene so maybe this is completely the wrong spot to ask.

(and yes I have been in stereo since ´90´s, wrote a 3D renderer in 1990, have an auto-stereoscopic monitor etc. so I´m not just some dumbo

[bobv5]

The one near the top of this page?

http://www.mtbs3d.com/phpBB/viewtopic.p ... &start=780" onclick="window.open(this.href);return false;

You could be in the right place.....

[pierreye]

Hi guys,
Completely new here. I was looking for the guy who posted the amazing HMZ-T1 wireless VR FPS gaming video on youtube. Uploaded by rjyap1 on Dec 27, 2011
I would like to hear how he build this setup or something similar for a blog I maintain. It´s focused on the European Tech scene so maybe this is completely the wrong spot to ask.

(and yes I have been in stereo since ´90´s, wrote a 3D renderer in 1990, have an auto-stereoscopic monitor etc. so I´m not just some dumbo

That's me! If you need a step by step video on the setup, I'll do that next 2 weeks when I have free time.

[mayaman]

Hi guys,
Completely new here. I was looking for the guy who posted the amazing HMZ-T1 wireless VR FPS gaming video on youtube. Uploaded by rjyap1 on Dec 27, 2011
I would like to hear how he build this setup or something similar for a blog I maintain. It´s focused on the European Tech scene so maybe this is completely the wrong spot to ask.

(and yes I have been in stereo since ´90´s, wrote a 3D renderer in 1990, have an auto-stereoscopic monitor etc. so I´m not just some dumbo

That's me! If you need a step by step video on the setup, I'll do that next 2 weeks when I have free time.

OMG yes please!!! This would be fantastic. I have the ability to get all the equipment but not the procedure of putting it all together. Please be as detailed as possible.

Thanks!

[fireslayer26]

Yes, a step by step video showing all parts/software/setup would be great!

ps- I can't find that 3.5" hdd holder that you are using.

[space123321]

Hi there - I am wondering if soemone can give me some pointers on Glovepie? Everytime I try and use the software, the wii remote freezes up, drifts in one direction and then shuts off? If I resynch the controller, I can run glovepie again however after 10-15 secs the same thing, the wii remote shuts itself off and looses synch?

I downloaded Wiimouse and it stays i synch with no issues whatsoever?

Thanks in advance for any help/tips... I am hoping to grab a HMZ-t1 in the future to get back into the HMD world!

EDIT - I figured it out, simply had to click "No auto bluetooth connect" in the trouble shooter options! Now off to synch up the PS3 Move items!

[space123321]

Hello again - sorry for the back and forths... now that I have the PS3 controllers synched and running in Windows, my Wii remote will now longer work? I put the remote into discovery mode and the PC no longers finds it? ANy ideas? Thanks again in advance!

EDIT - after some reading in this forum I realised that I would need another bluetooth dongle to get both to work. Thanks again for all the great info! I removed the Motioninjoy for now and the Wii remote is back up and running. Now to get my hands on an HMZ-t1!!!

[cybereality]

Yeah, you can get the PS3 and Wiimote to work at the same time, but I had to use a USB dongle (plus the built in one on my laptop). The Windows default bluetooth stack also sucks, and will disconnect the Wiimote for no reason all the time. So I was using the Toshiba bluetooth stack, and that helped.

[space123321]

Seems to be much more stable with the "no auto bluetooth connect" troubleshoot option picked - found this by trial and error (as I am using an onboard motherboard bluetooth) however have not had the wii remote drop since. I am very interested in the HMZ-t1 however no head tracking is what is really bothering me. I almost went ahead with the Vuzix 1200VR (as I have owned precious versions in the past) however I do want the Sony! The only thing I have seen online is the use of the TrackIR for head tracking on the Sony HMD. Has anyone worked a better VR solution?

[donkaradiablo]

If you need a step by step video on the setup, I'll do that next 2 weeks when I have free time.

That would be great! Most of us can't really build something around wireless hdmi and battery pack though. Having two videos would be greatly appreciated and would become the sticky around the globe for the VR enthusiast

1. Idiot's guide to VR including the following:

HMZ-T1 (how to setup stereo3D, audio over hdmi, only the quick bare essentials)
Wiimote + motionplus (for maximum compatibility with the software)
PS Move Navigator (to have more buttons available for use)
At least one recommendation for compatible and commercially available gun holder
Bluetooth connection (required hardware and setting it up)
GlovePie (setting it up)
Scripts (general FPS scripts and simple parameters that can be edited to one's taste)
Method (how to center view and stuff)

2. An advanced guide to turn the basic setup into a wireless VR fantasy
Wireless hdmi, battery pack, hdmi splitter, HMZ-T1 comfort mods, deeper script editing etc.

[donkaradiablo]

It may seem like a stupid question (yeah I'm a newbie) but what does a LOGITECH MX AIR Mouse strapped over a HMZ-T1 lack over the alternatives for FPS gaming? (besides the obvious fact that using a wireless gamepad for controls doesn't look 1% as cool as pierreye's setup and that pierreye's setup looks like it came out of a military training program)

Is a tracker like SpacePoint Fusion better than a gyro mouse for FPS games that only require 2DOF? Is the tracking quality better with a MotionPlus or a SpacePoint Fusion? Are there any practical challenges of using a gyro mouse for head tracking?

[pierreye]

Sorry that I procrastinate on writing the VR guide cause I can't pull myself out from Skyrim. Most likely the tutorial would be in PDF format as when I try to do a video tutorial, a lot of task involve in shooting, editing and dubbing (I sucks when talking while taking live video). Much easier to take some photo and type out the procedure. I had done some of the photo shooting and need sometime to write the tutorial.

As for comparison to other gyro, I used what I have on hand so can't compare which one is better. I prefer wiimote due to additional option and button such as roll and pitch function to detect the angle for additional command.

[donkaradiablo]

Better late than not as good as it might have been A step by step PDF might work better actually, if the link to that is given in the description of your youtube video. In the time it takes to make a half decent video, a much better looking document could be produced, making it the better choice for everyone



Anybody else willing to shed some light on the use of Logitech Mx Air mouse vs other alternatives for head tracking in FPS gaming? I'm thinking no need for other layers of software or emulation would make it a better candidate for the job but I have no first hand experience so I might be missing something. And the on-mouse "back" button that can be used to center view without any need for such an ingame keybinding should be a plus.



PS: Actually, unless I fail to foresee some practical challenges, it could be a good alternative for adding head tracking to pierreye's setup (but would require tweaking the GlovePie scripts to change the impact of the motionPlus on targeting, which I believe would be similar to the approach cybereality is taking)

[pierreye]

As promise, I setup the guide in google site and will update the project and script when I have time. Any suggestion are welcome.

https://sites.google.com/site/vrsetup/home" onclick="window.open(this.href);return false;

[space123321]

Awesome - thanks Pierreye!

[pierreye]

Preliminary hardware setup and configuration is up. If the instruction is not clear, do let me know. Also uploaded Dead Space 2 and Skyrim script.

[mayaman]

Incredible work buddy. You've achieved what huge corporations couldn't . Awesome stuff!

Thanks

[FingerFlinger]

Hey pierreye. First, your project is so exciting! I am so anxious to join in the fun, but it seems there is a long wait for these new HMDs...

I've been thinking lately about "soft" techniques we can use to increase immersion. I think we all want to decouple head movements from the reticle in first person games, but since it essentially isn't possible to mod that functionality into most game engines, what about tracking the head and gun separately, and switching between them? The trick is to define inflection points in the context of the game.

A couple of specific ideas about how to do this:

For an FPS, the mouse emulator would default to receiving input from the head tracker on your HMD. By mounting a switch at the back of the shoulder stock on your gun, you can identify when the player has entered combat and needs the mouse emulator to switch to a tracker on the gun. So, the gun tracker is the active input only when the player has raised it to his/her shoulder and is essentially looking down the barrel anyway.

This is next example is more relevant for an HMD user playing with mouse and keyboard.
For Skyrim, we can assume that when the player does not have a weapon drawn, they are not in combat, and want the headtracker to be the active input. When a weapon is drawn, they want the mouse to be the active input. I think we can determine this contextually.

A key on the keyboard is bound to the "sheathe weapon" command; I believe it's 'F' by default. 'F' will also unsheathe it. So we know that 'F' will toggle the current state of the weapon, and we can use this to switch between mouse and head tracker without making an imposition on the player.

(Furthermore, clicking the mouse when the weapon is sheathed will draw the weapon, but the left mouse click does many other things in the game, so how do we know that it means 'draw weapon' specifically? The other contexts in which mouse clicks are used is in dialogue or menus, and fortunately, most of these actions are associated with key presses. This means we can track the game state, and we can ignore mouse clicks that are made from within a menu. Although actually, I just realized that we want to revert control to the mouse anyway for menus and dialogue.)

Wow, sorry for the brain dump! I plan to prototype this kind of functionality myself once I acquire an HMD, but I thought these ideas might be useful to those of you who are already working on this stuff.

[pierreye]

The idea is great. I had experiment with 2 trackers, one for head and another for the rifle and it work out great.

In fact the latest Dead Space 2 script does include 2 wiimote tracker, one for the head and another on the gun devices. When you press the aim mode, I switch the mouse tracking from head to the wii sharpshooter. I going to shoot another video for Dead Space 2 and Skyrim coming weekend.

As for Skyrim, I not using the rifle. Left hand hold the PS3 navigator, right hand hold the wiimote. I only use head tracking for Skyrim but when you enter the menu, just hold PS3 navigator L1 button and the script will switch to right hand wiimote as mouse pointer. It's easier to pick a lock using right hand wiimote too instead of head tracking.

Just try out Kinect OPENNI skeleton tracking. Looks pretty good and able to detect hand/leg movement without facing front to the camera. I'm going to experiment with simple gesture first such as jump and crouch.

[FingerFlinger]

I'm looking forward to your new video.

Yes, I would think Skyrim is ideal for a wearable computing situation, since 'aiming' with your head feels natural with melee weapons.

Another big project that I'm looking at is a voice interface for game menus and dialogue. A voice interface removes the need to have an analog stick on hand, or to use your gun as a pointing device. I really feel like it would help the illusion if you always treat your gun as a gun, instead of "a gun, but sometimes a mouse". Maybe it's not that big of a deal, but if I can find a good voice recognition library, I think it would be worth trying.

OPENNI looks very cool. I wonder if it is precise enough for self-skinning. Meaning, if you look at your hand through the HMD, and it's wearing a leather gauntlet and tracking your actual movement.

[donkaradiablo]

Anybody willing to shed some light on the use of Logitech Mx Air mouse vs other alternatives for head tracking in FPS gaming? I'm thinking no need for other layers of software or emulation would make it a better candidate for the job but I have no first hand experience so I might be missing something. And the on-mouse "back" button that can be used to center view without any need for such an ingame keybinding should be a plus.

anyone?

[pierreye]

Here's 3rd person shooter VR using Dead Space 2 as example. This setup include head tracker using Motion Plus on the head. I need to re-shoot Skyrim video due to some problem to the picture quality.

http://www.youtube.com/watch?v=moYHZOhXTfk

[youtube-hd]http://www.youtube.com/watch?v=moYHZOhXTfk[/youtube-hd]

[cybereality]

Very cool.

[pierreye]

Here's Skyrim VR. A great game to try out in VR.

[youtube-hd]http://www.youtube.com/watch?v=jBP0q_kNdMY[/youtube-hd]

[CyberVillain]

Nice!, you can be a test case when FreePIE supports WiiMote

[pierreye]

Sure. No problem to be on the software testing team. Can't contribute much on programming nowadays as I stop for few years. Now only work as Project Manager getting specs, draft out the framework design and ensure my team deliver the application.

[fireslayer26]

Great work Pierreye!

[cybereality]

Cool.

[pierreye]

Pretty much any first person view/third person view games can be turn into a VR gaming with some scripting. Now only hope that developer would start to support dual mouse input, one for view and another for aiming cursor. This would push VR into next level.