Razer Hydra Support

[brantlew]

I've been playing around with the Hydra over the holidays and have just about completed FreePIE integration for it. I have a brief video that demonstrates Hydra position data being processed by FreePIE and sent to a custom Half Life 2 mod that I wrote that supports 6DOF head tracking. Unfortunately I am having problems getting accurate orientation data from the Hydra controller so instead of the Hydra, I am using the Hillcrest IMU sensor for orientation data. The sudden jerks in the video are caused by the Hillcrest, not the Hydra. So far the Hydra position data has been very smooth and reliable. Now if only I can get the correct orientation data... I have been trying to convert the quaternion data from the Hydra into Euler angles, but have only managed to get roll working. Yaw and pitch act very strangely. I have read that there are problems with the sixense dll and the size of the returned data structure which might cause byte misalignment. Does anybody have any experience with this problem or any sample code that I can reference?

[youtube]http://www.youtube.com/watch?v=Ad_3lMW1F3E[/youtube]

[StreetRat]

I did have some web page linked saved somewhere for some information, but cant find the project with the links in.

This is the type def from the sixense header file i have.

Code: Select all

typedef struct _sixenseControllerData {
  float pos[3];
  float rot_mat[3][3];
  float joystick_x;
  float joystick_y;
  float trigger;
  unsigned int buttons;
  unsigned char sequence_number;
  float rot_quat[4];
  unsigned short firmware_revision;
  unsigned short hardware_revision;
  unsigned short packet_type;
  unsigned short magnetic_frequency;
  int enabled;
  int controller_index;
  unsigned char is_docked;
  unsigned char which_hand;
  unsigned char hemi_tracking_enabled;
} sixenseControllerData;

then

Code: Select all

float angles[3] = {0,0,0};

void getHydraPYR(float* angles) {
	sixenseAllControllerData acd;
	for(int base=0; base < sixenseGetMaxBases(); base++) {
		sixenseSetActiveBase(base);
		sixenseGetAllNewestData(&acd);
		for(int cont=0; cont < sixenseGetMaxControllers(); cont++ ) {
			if(sixenseIsControllerEnabled(cont) && cont == 1) {
				float quat[4] = {0,0,0,0};
				quat[0] = acd.controllers[cont].rot_quat[0]; //x
				quat[1] = acd.controllers[cont].rot_quat[1]; //y
				quat[2] = acd.controllers[cont].rot_quat[2]; //z
				quat[3] = acd.controllers[cont].rot_quat[3]; //w

				// yaw
				angles[0] = radToDeg(atan2(2*quat[0]*quat[3]-2*quat[1]*quat[2], 1-2*pow(quat[0],2)-2*pow(quat[2],2)));
				// pitch
				angles[1] = radToDeg(atan2(2*quat[1]*quat[3]-2*quat[0]*quat[2], 1-2*pow(quat[1], 2)-2*pow(quat[2],2)));
				// roll
				angles[2] = radToDeg(asin(2*quat[0]*quat[1]+2*quat[2]*quat[3]));
			}
		}
	}
}

The codes not perfect, there are some small artifacts, but i havnt messed with it in a while so cant remember exactly what the problems were. Something to do with straight up and down

I have some working code in both vb.net and c++ i could send if your interested.

[brantlew]

Thanks StreetRat, I'll take a look at it. Right off the bat, I notice that you are assuming an [x, y, z, w] quaternion where I was assuming a [w, x, y, z].

[cybereality]

Looking cool, man!

Any reason you can't use quaternions in the Source engine? They should be more reliable than Euler angles (ie, to avoid gimbal lock).

[brantlew]

@StreetRat: Well I spent a whole afternoon messing around with the quaternion and ended up with the same formula you did. It all works pretty well except for "roll" which exhibits strange behavior after +-90 degrees. I can't decide if it's something strange about the hydra itself or the Euler conversion calculations. I do notice that even on the Sixense test apps I have a hard time rolling the controller around keeping the on-screen graphics aligned with the Z-axis. So maybe there is something inherently flawed with roll calculations and these magnetic devices ?? Whatever the reason, the orientation calculations are not as stable as IMU data. I think a sensor fusion using an IMU for orientation and the Hydra for position would be the best way to do head-tracking.

@cybereality: I might be able to, but I am specifically trying to test Euler angles in FreePIE.

[brantlew]

I checked in the Hydra code to GitHub. Everything seems to be working except some weird behavior on "roll" orientation which can be seen in the video. I don't know if its the Hydra itself or buggy calculations.

[youtube-hd]http://www.youtube.com/watch?v=WYJrN7lNNcc[/youtube-hd]

To use the Hydra, you should place the controllers on the base before you start the FreePIE script. The two controllers are accessed by index. You can determine which controller is on the left or right by checking the "side" value. Here is the script that I used in the demonstration video. (The "trackio" interface is not part of the FreePIE core and is a proprietary interface that I use for testing.)

Code: Select all

# Test script for sending Hydra tracker data via the TrackIO interface to a receiving application

trackio.yaw = hydra[0].yaw
trackio.pitch = hydra[0].pitch
trackio.roll = hydra[0].roll

trackio.x = -hydra[0].x * 0.2
trackio.z = hydra[0].y * 0.2
trackio.y = -hydra[0].z * 0.2

keyboard.setKey(Key.W, hydra[0].joyy > 0.5)
keyboard.setKey(Key.S, hydra[0].joyy < -0.5)
keyboard.setKey(Key.D, hydra[0].joyx > 0.5)
keyboard.setKey(Key.A, hydra[0].joyx < -0.5)

[CyberVillain]

Nice work!

Code: Select all

      public HydraPlugin() {
         Controller[0] = new Sixense.ControllerData();
         Controller[1] = new Sixense.ControllerData();

         Angles[0] = new Sixense.ControllerAngles();
         Angles[1] = new Sixense.ControllerAngles();
      }

Maybe this should be moved to the Start method? All plugins are invoked at FreePIE startup to populate the settings, this will invoke the Hydra SDK dll for everybody even people that are not using the specific hardware. If its moved to the Start method it it will only invoke it for those that have scripts thats using it..

[brantlew]

Maybe this should be moved to the Start method?

Good catch. Will do...

[brantlew]

Here is a reference for all the available fields.

Code: Select all

side ('L', 'R', '?')    - which side is the controller on
one (true, false)       - 1 button
two (true, false)       - 2 button
three (true, false)     - 3 button
four (true, false)      - 4 button
start (true, false)     - start button
bumper (true, false)    - front bumper button
trigger (-1.0 - 1.0)    - front trigger
joybutton (true, false) - front bumper button
joyx(-1.0 - 1.0)        - thumbstick left/right
joyy (-1.0 - 1.0)       - thumbstick forward/back
x (-val - +val)         - left/right position
y (-val - +val)         - up/down position
z (-val - +val)         - forward back position
yaw (-PI - +PI)         - clockwise yaw
pitch (-PI - +PI)       - pitch
roll (-PI - +PI)        - clockwise roll
q0 (-1.0 - 1.0)         - quaternion
q1 (-1.0 - 1.0)         - quaternion
q2 (-1.0 - 1.0)         - quaternion
q3 (-1.0 - 1.0)         - quaternion

[CyberVillain]

Do you have anything in the pipe? I'm planning a binary release later today

[brantlew]

No I got everything in.

[Alesh]

Hi, could somebody please help me out with Hydra in Half Life 2 integration? I'm afraid I still don't quite get it.
I have a DIY rift up and ready with the right Hydra stick strapped to it. I also have FreePIE and Vireo Perception running.
I'm running this code in freePIE:

Code: Select all

def update():
    global yaw
    yaw = hydra[1].yaw
    global pitch
    pitch = -hydra[1].roll
    global roll
    roll = hydra[1].pitch

if starting:
    centerYaw = 0
    centerPitch = 0
    centerRoll = 0
   
    yaw = 0
    pitch = 0
    roll = 0
    android.update += update

freeTrack.yaw  = yaw - centerYaw
freeTrack.pitch = pitch - centerPitch
freeTrack.roll = roll - centerRoll

diagnostics.watch(hydra[1].pitch)
diagnostics.watch(hydra[1].yaw)
diagnostics.watch(hydra[1].roll)

if keyboard.getKeyDown(Key.LeftControl) and keyboard.getPressed(Key.C):
    centerYaw = yaw
    centerPitch = pitch
    centerRoll = roll

The diagnostics seems fine as the values a changing as I move the Hydra around so connection to FreePIE seems to be working. But when I run HL2 the Hydra pitch/jaw/roll controls don't seem to work. Of course I am running a Vireio Perception as well with DIY Rift/Free Track setup. I thought the Hydra's jaw/pitch/roll are feeding the freeTrack jaw/pitch/roll values. What am I missing? should I use vireioSMT instead of freeTrack? Please note that I'm not a programer so if you could also include the simple (obvious) steps that would be awesome! Thanks a bunch.

[Alesh]

Ok, tried it with vireioSMT:

Code: Select all

vireioSMT.yaw  = yaw - centerYaw
vireioSMT.pitch = pitch - centerPitch
vireioSMT.roll = roll - centerRoll

still doesn't work...
I think I may have a problem with the line

Code: Select all

android.update += update

I can't seem to convert this line so it's appropriate for Hydra, for example:

Code: Select all

hydra.update += update

doesn't work. I get this error: 'Array[HydraPluginGlobal]' object has no attribute 'update'

[CyberVillain]

doesn't work. I get this error: 'Array[HydraPluginGlobal]' object has no attribute 'update'

Hydra plugin is written by Brantlew and he favors the poll model instead of the event model, since I dont own a Hydra I cant change his code since I cant test if it works. So you have to move that part from the update function to the main loop. Like

Code: Select all

if starting:
    centerYaw = 0
    centerPitch = 0
    centerRoll = 0

yaw = hydra[1].yaw
pitch = -hydra[1].roll
roll = hydra[1].pitch

vireioSMT.yaw  = yaw - centerYaw
vireioSMT.pitch = pitch - centerPitch
vireioSMT.roll = roll - centerRoll

diagnostics.watch(hydra[1].pitch)
diagnostics.watch(hydra[1].yaw)
diagnostics.watch(hydra[1].roll)

if keyboard.getKeyDown(Key.LeftControl) and keyboard.getPressed(Key.C):
    centerYaw = yaw
    centerPitch = pitch
    centerRoll = roll

[Alesh]

Ah, that makes more sense now! Thanks a bunch, CyberVillain. Now I see I have to convert the radians that Hydra spits out to degrees. I guess this would be the best way to do it:

Code: Select all

yaw = ((hydra[1].yaw / math.pi) * 180);
   pitch = ((hydra[1].pitch / math.pi) * 180);
   roll = ((hydra[1].pitch / math.pi) * 180)

Also I see I get quite a bit of noise in the game when I keep my head/hydra controller still. I would probably need to run a filter on the Hydra pitch/yaw/roll stream, wouldn't I? Is this something you would use:

Code: Select all

deltaYaw = filters.delta(yaw)
 deltaPitch = filters.delta(pitch)
 deltaRoll = filters.delta(pitch)

Not sure what this delta does though. I've seen it in other code examples and it seems to do the filtering. Is this the Kalman filter that seems to be used in the tracking devices quite a bit?

[CyberVillain]

you can use math.degrees(hydra[1].yaw) which does the same thing for you

delta just returns the delta between two samples, its not what you want todo.

use filters.simple, its a simple filters the second argument is a factor how much of the previous sample you should use so

Code: Select all

filters.simple(hydra[1].yaw, 0.8)

will use 80% of the old sample and 20% of the new sample

My plan was to introduce more filters but I never got to it

edit: Kalman filter is used to fuse data from different kind of units like velocity (Gyro) and acceleration (Accelerator) into x, y, z, w (Quaternion), that can me used for orientation of a device

[Alesh]

Brilliant! Thank you for making things more clear to me, I didn't even know about the filters.simple. Now I just need to figure out which pitch/jaw/roll multipliers I need to get a 1:1 head movement in HL2 and I'm good to go!

Will post the code here once it's done in case someone else might find it useful.

[Alesh]

There you go, for all the noobs like me. Hopefully it will save someone hours I spent looking around to make this stuff work.
The freePIE code below seems to work nicely for Half Life 2. Mind you this is a setup for my DIY rift, where my right Hydra stick is strapped to the HMD while the joystick on the left stick is used as a AWSD key stroke to move the character around the game environment.

Code: Select all

if starting:
    centerYaw = 0
    centerPitch = 0
    centerRoll = 0

yaw = filters.simple(math.degrees(hydra[1].yaw),0.2)/1.2
pitch = filters.simple(math.degrees(hydra[1].pitch),0.2)/1.2
roll = filters.simple(math.degrees(hydra[1].roll),0.2)

vireioSMT.yaw  = (yaw - centerYaw)
vireioSMT.pitch = (pitch - centerPitch)
vireioSMT.roll = (roll - centerRoll)

if keyboard.getKeyDown(Key.LeftControl) and keyboard.getPressed(Key.C):
    centerYaw = yaw
    centerPitch = pitch
    centerRoll = roll
    
keyboard.setKey(Key.W, hydra[0].joyy > 0.5)
keyboard.setKey(Key.S, hydra[0].joyy < -0.5)
keyboard.setKey(Key.D, hydra[0].joyx > 0.5)
keyboard.setKey(Key.A, hydra[0].joyx < -0.5)

After you copy the script above to freePie, run it by pressing F5. Now all you have to do is run Vireo Perception Shared Memory Tracker mode and then run the supported game. Voila! If you get some weird camera rotations in the game (like your view being tilted) press Ctrl + c to center/reset the rotations.

Like I said this are my first lines of code in freePIE, so if anyone feels like adding/optimizing anything, please do so!

[CyberVillain]

Nice work! 0.2 took care of the noise?

[Alesh]

Actually I'm not sure if the noise wasn't the result of Hydra being close to display and controller - it was strapped to my HMD so there may be some interference. 0.2 seems to work ok. I started with 0.8 but I felt that it introduced a bit of latency in the head movement. I guess it makes sense if this only takes the 20% of the new value there has to a tiny lag.

Glad I got it working - thanks to you that is! I just recieved my YEI-3 space tracker, so the next stop would be to replace the code to make it work with YEI for the headtracking and Hydra for moving around, jumping, crouching etc.. Will post the working code in the YEI-3 thread. Looking forward to take some weight off my DIY rift . That chip is really small

[Croccy22]

Does this plugin support write as well as read?

I see they now have the Hydra supported in Minecrift so it would be nice if I could now emulate it using my AHRS IMU and a 360 controller or something?

Matt.

[CyberVillain]

Sorry no, only read. Minecraft is written in Java so I doubt it uses the sixense.dll?

sixense.dll is written in C so it would be possible to spoof it with FreePIE like we do with TrackIR. (If someone took the time)

One of the guys from minecraft was talking about supporting FreePIE but I do not know how far they got

[Croccy22]

Arrgghh, All I want is Minecraft! hehe

Mouse emulation works but I want head roll

Matt.

PS. Thanks for the reply!

[Alesh]

I would like to setup my Hydra controllers for Portal 2. I know how to assign keys to hydra controls, like:

Code: Select all

keyboard.setKey(Key.W, hydra[0].joyy > 0.5)
keyboard.setKey(Key.S, hydra[0].joyy < -0.5)
keyboard.setKey(Key.D, hydra[0].joyx > 0.5)
keyboard.setKey(Key.A, hydra[0].joyx < -0.5)

How to tare my YEI 3-space sensor with a bumper button:

Code: Select all

if hydra[0].bumper == 1:
	yei[0].tareSensor()

But could somebody please give me an example of how would I assign the left and right mouse button to a Hydra control. I would need that to assign the blue and orange portal firing mode. I know how to do it through Hydra's Motion creator 2 editor, but I would like to learn the freePIE setup as well. Thank you!

[Alesh]

Hmm, tried this:

Code: Select all

if hydra[0].trigger == 1:
	mouse.leftButton = 1

But when I press the trigger my portal gun goes into auto fire mode
I feel like I'm almost there..

[Alesh]

Ah, silly me

Code: Select all

if hydra[0].trigger == 1:
	mouse.leftButton = 1
	mouse.leftButton = 0

This works. If someone has a more optimized way to do it, please let me know..

[CyberVillain]

Code: Select all

mouse.leftButton = hydra[0].trigger

[Alesh]

Code: Select all

mouse.leftButton = hydra[0].trigger

Funny, I thought I tried that... Must have typed something wrong. Thank you!