Head Tracking/Gun controller help and suggestions required!!

[fireslayer26]

Yes, a step by step video showing all parts/software/setup would be great!

ps- I can't find that 3.5" hdd holder that you are using.

[space123321]

Hi there - I am wondering if soemone can give me some pointers on Glovepie? Everytime I try and use the software, the wii remote freezes up, drifts in one direction and then shuts off? If I resynch the controller, I can run glovepie again however after 10-15 secs the same thing, the wii remote shuts itself off and looses synch?

I downloaded Wiimouse and it stays i synch with no issues whatsoever?

Thanks in advance for any help/tips... I am hoping to grab a HMZ-t1 in the future to get back into the HMD world!

EDIT - I figured it out, simply had to click "No auto bluetooth connect" in the trouble shooter options! Now off to synch up the PS3 Move items!

[space123321]

Hello again - sorry for the back and forths... now that I have the PS3 controllers synched and running in Windows, my Wii remote will now longer work? I put the remote into discovery mode and the PC no longers finds it? ANy ideas? Thanks again in advance!

EDIT - after some reading in this forum I realised that I would need another bluetooth dongle to get both to work. Thanks again for all the great info! I removed the Motioninjoy for now and the Wii remote is back up and running. Now to get my hands on an HMZ-t1!!!

[cybereality]

Yeah, you can get the PS3 and Wiimote to work at the same time, but I had to use a USB dongle (plus the built in one on my laptop). The Windows default bluetooth stack also sucks, and will disconnect the Wiimote for no reason all the time. So I was using the Toshiba bluetooth stack, and that helped.

[space123321]

Seems to be much more stable with the "no auto bluetooth connect" troubleshoot option picked - found this by trial and error (as I am using an onboard motherboard bluetooth) however have not had the wii remote drop since. I am very interested in the HMZ-t1 however no head tracking is what is really bothering me. I almost went ahead with the Vuzix 1200VR (as I have owned precious versions in the past) however I do want the Sony! The only thing I have seen online is the use of the TrackIR for head tracking on the Sony HMD. Has anyone worked a better VR solution?

[donkaradiablo]

If you need a step by step video on the setup, I'll do that next 2 weeks when I have free time.

That would be great! Most of us can't really build something around wireless hdmi and battery pack though. Having two videos would be greatly appreciated and would become the sticky around the globe for the VR enthusiast

1. Idiot's guide to VR including the following:

HMZ-T1 (how to setup stereo3D, audio over hdmi, only the quick bare essentials)
Wiimote + motionplus (for maximum compatibility with the software)
PS Move Navigator (to have more buttons available for use)
At least one recommendation for compatible and commercially available gun holder
Bluetooth connection (required hardware and setting it up)
GlovePie (setting it up)
Scripts (general FPS scripts and simple parameters that can be edited to one's taste)
Method (how to center view and stuff)

2. An advanced guide to turn the basic setup into a wireless VR fantasy
Wireless hdmi, battery pack, hdmi splitter, HMZ-T1 comfort mods, deeper script editing etc.

[donkaradiablo]

It may seem like a stupid question (yeah I'm a newbie) but what does a LOGITECH MX AIR Mouse strapped over a HMZ-T1 lack over the alternatives for FPS gaming? (besides the obvious fact that using a wireless gamepad for controls doesn't look 1% as cool as pierreye's setup and that pierreye's setup looks like it came out of a military training program)

Is a tracker like SpacePoint Fusion better than a gyro mouse for FPS games that only require 2DOF? Is the tracking quality better with a MotionPlus or a SpacePoint Fusion? Are there any practical challenges of using a gyro mouse for head tracking?

[pierreye]

Sorry that I procrastinate on writing the VR guide cause I can't pull myself out from Skyrim. Most likely the tutorial would be in PDF format as when I try to do a video tutorial, a lot of task involve in shooting, editing and dubbing (I sucks when talking while taking live video). Much easier to take some photo and type out the procedure. I had done some of the photo shooting and need sometime to write the tutorial.

As for comparison to other gyro, I used what I have on hand so can't compare which one is better. I prefer wiimote due to additional option and button such as roll and pitch function to detect the angle for additional command.

[donkaradiablo]

Better late than not as good as it might have been A step by step PDF might work better actually, if the link to that is given in the description of your youtube video. In the time it takes to make a half decent video, a much better looking document could be produced, making it the better choice for everyone



Anybody else willing to shed some light on the use of Logitech Mx Air mouse vs other alternatives for head tracking in FPS gaming? I'm thinking no need for other layers of software or emulation would make it a better candidate for the job but I have no first hand experience so I might be missing something. And the on-mouse "back" button that can be used to center view without any need for such an ingame keybinding should be a plus.



PS: Actually, unless I fail to foresee some practical challenges, it could be a good alternative for adding head tracking to pierreye's setup (but would require tweaking the GlovePie scripts to change the impact of the motionPlus on targeting, which I believe would be similar to the approach cybereality is taking)

[pierreye]

As promise, I setup the guide in google site and will update the project and script when I have time. Any suggestion are welcome.

https://sites.google.com/site/vrsetup/home" onclick="window.open(this.href);return false;

[space123321]

Awesome - thanks Pierreye!

[pierreye]

Preliminary hardware setup and configuration is up. If the instruction is not clear, do let me know. Also uploaded Dead Space 2 and Skyrim script.

[mayaman]

Incredible work buddy. You've achieved what huge corporations couldn't . Awesome stuff!

Thanks

[FingerFlinger]

Hey pierreye. First, your project is so exciting! I am so anxious to join in the fun, but it seems there is a long wait for these new HMDs...

I've been thinking lately about "soft" techniques we can use to increase immersion. I think we all want to decouple head movements from the reticle in first person games, but since it essentially isn't possible to mod that functionality into most game engines, what about tracking the head and gun separately, and switching between them? The trick is to define inflection points in the context of the game.

A couple of specific ideas about how to do this:

For an FPS, the mouse emulator would default to receiving input from the head tracker on your HMD. By mounting a switch at the back of the shoulder stock on your gun, you can identify when the player has entered combat and needs the mouse emulator to switch to a tracker on the gun. So, the gun tracker is the active input only when the player has raised it to his/her shoulder and is essentially looking down the barrel anyway.

This is next example is more relevant for an HMD user playing with mouse and keyboard.
For Skyrim, we can assume that when the player does not have a weapon drawn, they are not in combat, and want the headtracker to be the active input. When a weapon is drawn, they want the mouse to be the active input. I think we can determine this contextually.

A key on the keyboard is bound to the "sheathe weapon" command; I believe it's 'F' by default. 'F' will also unsheathe it. So we know that 'F' will toggle the current state of the weapon, and we can use this to switch between mouse and head tracker without making an imposition on the player.

(Furthermore, clicking the mouse when the weapon is sheathed will draw the weapon, but the left mouse click does many other things in the game, so how do we know that it means 'draw weapon' specifically? The other contexts in which mouse clicks are used is in dialogue or menus, and fortunately, most of these actions are associated with key presses. This means we can track the game state, and we can ignore mouse clicks that are made from within a menu. Although actually, I just realized that we want to revert control to the mouse anyway for menus and dialogue.)

Wow, sorry for the brain dump! I plan to prototype this kind of functionality myself once I acquire an HMD, but I thought these ideas might be useful to those of you who are already working on this stuff.

[pierreye]

The idea is great. I had experiment with 2 trackers, one for head and another for the rifle and it work out great.

In fact the latest Dead Space 2 script does include 2 wiimote tracker, one for the head and another on the gun devices. When you press the aim mode, I switch the mouse tracking from head to the wii sharpshooter. I going to shoot another video for Dead Space 2 and Skyrim coming weekend.

As for Skyrim, I not using the rifle. Left hand hold the PS3 navigator, right hand hold the wiimote. I only use head tracking for Skyrim but when you enter the menu, just hold PS3 navigator L1 button and the script will switch to right hand wiimote as mouse pointer. It's easier to pick a lock using right hand wiimote too instead of head tracking.

Just try out Kinect OPENNI skeleton tracking. Looks pretty good and able to detect hand/leg movement without facing front to the camera. I'm going to experiment with simple gesture first such as jump and crouch.

[FingerFlinger]

I'm looking forward to your new video.

Yes, I would think Skyrim is ideal for a wearable computing situation, since 'aiming' with your head feels natural with melee weapons.

Another big project that I'm looking at is a voice interface for game menus and dialogue. A voice interface removes the need to have an analog stick on hand, or to use your gun as a pointing device. I really feel like it would help the illusion if you always treat your gun as a gun, instead of "a gun, but sometimes a mouse". Maybe it's not that big of a deal, but if I can find a good voice recognition library, I think it would be worth trying.

OPENNI looks very cool. I wonder if it is precise enough for self-skinning. Meaning, if you look at your hand through the HMD, and it's wearing a leather gauntlet and tracking your actual movement.

[donkaradiablo]

Anybody willing to shed some light on the use of Logitech Mx Air mouse vs other alternatives for head tracking in FPS gaming? I'm thinking no need for other layers of software or emulation would make it a better candidate for the job but I have no first hand experience so I might be missing something. And the on-mouse "back" button that can be used to center view without any need for such an ingame keybinding should be a plus.

anyone?

[pierreye]

Here's 3rd person shooter VR using Dead Space 2 as example. This setup include head tracker using Motion Plus on the head. I need to re-shoot Skyrim video due to some problem to the picture quality.

http://www.youtube.com/watch?v=moYHZOhXTfk

[youtube-hd]http://www.youtube.com/watch?v=moYHZOhXTfk[/youtube-hd]

[cybereality]

Very cool.

[pierreye]

Here's Skyrim VR. A great game to try out in VR.

[youtube-hd]http://www.youtube.com/watch?v=jBP0q_kNdMY[/youtube-hd]

[CyberVillain]

Nice!, you can be a test case when FreePIE supports WiiMote

[pierreye]

Sure. No problem to be on the software testing team. Can't contribute much on programming nowadays as I stop for few years. Now only work as Project Manager getting specs, draft out the framework design and ensure my team deliver the application.

[fireslayer26]

Great work Pierreye!

[cybereality]

Cool.

[pierreye]

Pretty much any first person view/third person view games can be turn into a VR gaming with some scripting. Now only hope that developer would start to support dual mouse input, one for view and another for aiming cursor. This would push VR into next level.

[WiredEarp]

Now only hope that developer would start to support dual mouse input, one for view and another for aiming cursor.

I'd rather hope that they start to support dual input of multiple types, not just dual mouse input. Tying it to mice would mean its not absolute, and thats where its at for real VR imho.

[pierreye]

That's true but the problem is no standard input device for absolute view which make it difficult to support.

I had seen a demo on HL2 mod that support mouse aiming crosshair and independent view from sixense.

[CyberVillain]

Joysticks are absolute, the problem is that most games use them in a relative manner

[cybereality]

The main issue is that developers always want to support the bottom-line, and for 99% of customers that means a mouse and keyboard. Even when better options exist (ie Razer Hydra) there usually isn't enough of a critical mass to justify support. Even though adding support could take only a couple of days (or less) developers don't seem to think its worth it. Not sure that will change any time soon unless there was a massively popular VR peripheral for the PC.

[WiredEarp]

I dont think most joystick ranges have sufficient precision, compared to things like TrackIR etc. Most potentiometer ones only have 1024 resolution, so even if you wrapped the coordinates into them, the accuracy would not be as great. Of course, if we are mapping into a virtual joystick, we could possibly increase the precision to any level.

I'm thankful that most games use joysticks relatively, don't know if you remember the hell of Apple 2 joysticks when heaps of game developers wrote them using absolute addressing instead of relative.
Having to put the joystick 1/4 of the way so my spaceship is 1/4 of the way across the screen? lame!!!

I think half the problem is the lack of an easy interface. Sure, we have VRPN, but its not well known or used. What is really needed is just a very simple generic input interface, that game devs can read, and use in their games. Then, other systems can just plug into this. I guess FreePIE could be a contender here - what is needed is NOT 'compatibility with TrackIR, Hydra, etc' but just 'generic input' compatibility, so that game devs can just read the interface, and modify their view/aiming appropriately.

[CyberVillain]

I just stated that there already is a general software interface that is absolute... DirectX has no limitations when it comes to precision... Its up to the hardware, or emulation in this case.... The problem is still that game developers use Joysticks in a relative manner, this is sad otherwise you could use ppJoy as a bridge between lets say FreeTrack and any game....

[WiredEarp]

Ok, so even if the precision isn't a problem, and we can use virtual joysticks with multiple axis to represent the different points, a standard for which axis represents what and what ranges are allowed, etc, would still be required (or at least, be very handy). This would allow configuration to be set once for each device, and after that, everything would work smoothly, with every game.

[CyberVillain]

Well ranges are not a problem, DX has range from -1 to 1 so your FreePIE or GlovePIE script just has to convert 360 degrees to that range, but, since the game takes that value and uses it reletive there is no point :/

The games does x += value instead of doing x = value

[cybereality]

If you could somehow "know" what the angle is in the game, then I think you could hack it so the joystick was absolute without changing anything in the game. This is one of the things I hope to accomplish with my DirectX driver (if I ever pull it off).

[CyberVillain]

Some kind of memory hack? Then you need to know the address for each game :/
Are going to write directly to the memory or just read the value and set it using conventional joystick / mouse emulating?

If your doing the first changes are that online games will ban you with punkbuster

It would be really cool if you pull this off

[cybereality]

Well right now I am just planning on getting the camera orientation via DirectX. We can probably assume most every game uses the same up vector, so this should be helpful in re-centering the pitch. In terms of knowing which way is forward (for yaw), this may be harder (and/or impossible) but I think it could be done with some work. And this would most certainly be considered a hack, and get you banned online. I have no doubts about that. But I mostly play single-player so I don't mind.

[pierreye]

Another issue to solve is when you hit the limit of the pitch, you can keep on increasing DirectInput Y value but the coordinate just doesn't move. This is where I hit the hurdle for auto centering script.

[brantlew]

If you could somehow "know" what the angle is in the game, then I think you could hack it so the joystick was absolute without changing anything in the game. This is one of the things I hope to accomplish with my DirectX driver (if I ever pull it off).

Sounds interesting but tricky. You would need to determine on a per-game basis the scaling between joystick ticks and turning speed. Once you have that rate you could sort of hack absolute motion by simulating a joystick push and release. For example, if your absolute yaw went from 0 degrees to 45 degrees in 100 milliseconds you would have to simulate a joystick push of a magnitude that caused a 450 degree per second turn and then recenter the simulated joystick after 100 ms.

A couple things that could be problematic. There may be a limit to the turning speed supported in the game in which case some motions might feel a bit laggy. Also I wonder how the joystick polling rate would tie in. It might be difficult to pin down an exact joystick turning rate if it is dependent on the frame rate. Different areas of the game might have different turning behavior.

Still, it would be an interesting exercise. For yaw, I don't think it would matter if your calculations were off. So what if your real world yaw is out of sync with your in-game yaw? Pitch errors would be noticeable though.

[PalmerTech]

Yeah, as long as the yaw matches up to what you are seeing during the turn, it does not matter if it eventually drifts off a bit.

[CyberVillain]

I dont think you can use Joysticks in most games today Brantlew, the game implementation is just too slow.. You'll have to use the mouse :/

I wished all games had a switch for relative or absolute joystick use :/