FreePIE official thread

Post by **BadKarma** » Thu May 17, 2012 7:30 am

brantlew wrote:I got bad news for you BadKarma. There is very little you can do to solve drift with accelerometers. They are inherently unstable devices and you should expect some drift if you use them. The only way to really fix them is with sensor fusion which basically means dynamically recalibrating them with other sensors. Often magnetometers and compasses are used to recalibrate but those are also a bit unstable because of electromagnetic field noise - so it really is a big problem. Sometimes you can "hack" it by programming a calibrate button that re-centers everything when you press it.

You're right, I know it's a big problem and it cannot be 'fixed'. It's not my goal to create a perfect tracker with this, as I know it's most likely impossible without extra calibration sensors or some genius algorithm I'll never be able to conjure up

.

I must say though that the AHRS firmware uses a nice sensor fusion algorithm to begin with (check out this small paragraph with more referrals to scientific papers, if you haven't already). The AHRS firmware already uses the magnetometer and gyro to compensate for drifting, and after calibrating these especially I've come up with some pretty good results (video is uploading as we speak!). And yes, I'll most likely implement a recalibration button, it seems to be my only option since I do not have all the time in the world to wrap this project up..

Anyways, still 30 min. to upload completion, so you be the judge then

just be kind.. hehe.

Post by **BadKarma** » Thu May 17, 2012 8:09 am

Alright, here's the video finally. Let me know what you guys think (and be brutally honest, since that's way more helpful

)

[youtube]http://www.youtube.com/watch?v=Toz7-XSnHXk[/youtube]

Post by **brantlew** » Thu May 17, 2012 9:05 am

Looks great! I am glad you don't have the drift problems that CyberVillain had. It proves that this device is a viable head-tracker. Maybe he had a bad part or an electromagnetically noisy environment?

One other thought. It is possible that the drift you notice is due to mouse emulation. I am not 100% sure of the causes but I think the frame-rate/mouse sampling rate and the rate of mouse input commands can get out of sync and cause the type of problems you are seeing. Since the mouse movements are cumulative (as opposed to absolute), if the game misses some of them it will drift. Also if the game uses some type of mouse-acceleration algorithm where quick movements cause greater angular motion then you will have this problem as well.

Have you verified that the raw data angles are getting misaligned or is it just the "north" direction in the game?

Post by **BadKarma** » Thu May 17, 2012 11:35 am

brantlew wrote:Looks great! I am glad you don't have the drift problems that CyberVillain had. It proves that this device is a viable head-tracker. Maybe he had a bad part or an electromagnetically noisy environment?

Most likely.. Although judging from his last videos he had some yaw problems as well? Not sure if that had anything to do with his additional coding as he tried to get rid of his drifting though..

brantlew wrote:One other thought. It is possible that the drift you notice is due to mouse emulation. I am not 100% sure of the causes but I think the frame-rate/mouse sampling rate and the rate of mouse input commands can get out of sync and cause the type of problems you are seeing. Since the mouse movements are cumulative (as opposed to absolute), if the game misses some of them it will drift. Also if the game uses some type of mouse-acceleration algorithm where quick movements cause greater angular motion then you will have this problem as well.

Thought about the mouse acceleration as well. Going to check how this works in UDK, since I probably know how to disable any mouse acceleration used in the game. Not so familiar with Source engine (which Left4Dead2 uses).

brantlew wrote:Have you verified that the raw data angles are getting misaligned or is it just the "north" direction in the game?

Not sure how I would go about verifying this? Perhaps turning on the debugging function in FreePIE to see the raw angles and check the response ingame would do the trick.. And what do you mean with the north direction?

Post by **brantlew** » Thu May 17, 2012 12:34 pm

BadKarma wrote:Not sure how I would go about verifying this? Perhaps turning on the debugging function in FreePIE to see the raw angles and check the response ingame would do the trick.. And what do you mean with the north direction?

Yes, you would have to use the FreePie debugging functions. Something like "diagnostics:debug(sensor:getYaw());" Unfortunately the FreePIE output console is not optimized to print quickly so it is difficult to debug with. You should try to limit your debug output to every 10th or 20th cycle to avoid overloading the GUI. I always compile a console-only version of FreePIE to avoid these printing issues.

"north" just meant any fixed direction in the game.

Post by **CyberVillain** » Thu May 17, 2012 2:03 pm

It looks way better than mine! I think its very hard to remove drift completely with mouse emulation. Will you hook it up to a rifle of some sort?

Post by **CyberVillain** » Thu May 17, 2012 2:05 pm

brantlew wrote:
BadKarma wrote:Not sure how I would go about verifying this? Perhaps turning on the debugging function in FreePIE to see the raw angles and check the response ingame would do the trick.. And what do you mean with the north direction?
Yes, you would have to use the FreePie debugging functions. Something like "diagnostics:debug(sensor:getYaw());" Unfortunately the FreePIE output console is not optimized to print quickly so it is difficult to debug with. You should try to limit your debug output to every 10th or 20th cycle to avoid overloading the GUI. I always compile a console-only version of FreePIE to avoid these printing issues.

"north" just meant any fixed direction in the game.

I will add a Watch feature "soon". So you will do diagnostics:watch(sensor:getYaw()) and then it will be presented in a grid with name sensor:getYaw() and the current value

Post by **cybereality** » Thu May 17, 2012 6:24 pm

Not too shabby. Some drift, sure, but the signal looks nice and clean at least.

Post by **CyberVillain** » Fri May 18, 2012 9:40 am

By the way, in the clip when you aim for the bird you hit the roof of what the Source engine let you aim upward, but you continue to move the device up for e while even when the camera does not longer move. This will make the Y-axis out fo sync because when you start to move down again the camera will move directly

Post by **BadKarma** » Fri May 18, 2012 4:03 pm

CyberVillain wrote:By the way, in the clip when you aim for the bird you hit the roof of what the Source engine let you aim upward, but you continue to move the device up for e while even when the camera does not longer move. This will make the Y-axis out fo sync because when you start to move down again the camera will move directly

You know, I actually laughed a bit when I read this! It's so obvious yet I hadn't thought about that.. Thanks for pointing that out to me CV. Been working on a lot of paperwork today but will experiment further tomorrow with the Bluetooth module and see if I can work around the mouse restrictions in UDK. I'll run the diagnostics and compare them to the in-game results like brantlew suggested. Might even be Sunday when I do this though, but I'll keep in touch!

- Cheers

Post by **CyberVillain** » Sat May 19, 2012 5:50 am

Added watch functionality, but will upload binary first when Intellisense is done
You can however download the source and build the binary yourself if you have VS

Post by **BadKarma** » Sat May 19, 2012 10:28 am

Oh nice! Could definitely use that output

Yeah, I have VS installed. Will download the binary and see if I can compile it. Should I be aware of any special settings that need to be on or off for compiling? Just asking since the last time I compiled anything was over half a year, haha..

Post by **brantlew** » Sat May 19, 2012 1:13 pm

cool. nice feature

Post by **CyberVillain** » Sat May 19, 2012 3:28 pm

@BadKarma I think that brantlew had some problem with the Express version of VS, but I'm not sure, he can answer that better. If you have any of the paid versions it should work out of the box

@Brantlew, thanks man, been thinking of doing the feature for a long time, but haven't found the motivation until now

Post by **BadKarma** » Sun May 20, 2012 3:39 pm

Quick update on the problems I was having. It turns out you were totally right CyberVillain! I managed to disable the vertical rotation limit in UDK by simply deleting the lines of code that were causing the limit in the first place. Not sure if any of you guys ever scripted in UDK, but I'll post it none the less

Original code in UTPlayercontroller:

Code: Select all

event Rotator LimitViewRotation( Rotator ViewRotation, float ViewPitchMin, float ViewPitchMax )
{
	ViewRotation.Pitch = ViewRotation.Pitch & 65535;

    if( ViewRotation.Pitch > ViewPitchMax &&
		ViewRotation.Pitch < (65535+ViewPitchMin) )
	{
		if( ViewRotation.Pitch < 32768 )
		{
			ViewRotation.Pitch = ViewPitchMax;
		}
		else
		{
			ViewRotation.Pitch = 65535 + ViewPitchMin;
		}
	}

	return ViewRotation;
}

Stripped code:

Code: Select all

event Rotator LimitViewRotation( Rotator ViewRotation, float ViewPitchMin, float ViewPitchMax )
{
	return ViewRotation;
}

This only works partially though, as my controls seem to invert as soon as I pass the top limit. This is only normal it seems (did some quick searching) as I would need to calculate my rotation using quaternions to get a correct rotation. BUT, for my project (a tracker on a HMD) it will work just fine, since no human can lift their head high enough and look behind themselves (except perhaps in Horror movies

). So for now, this will do. I set the multiplier in the FreePIE script to 5 by the way and the results are much better.

Anyway, no video as I didn't deem it necessary for now. Tomorrow I will work some more on the casing for the Razor, using a 3D printer. I'll post a couple of pics once it's all done

- Cheers

Post by **CyberVillain** » Mon May 21, 2012 12:33 am

Nice that it works better. Since you can set the pitch and yaw from code isnt there wanyway to pass the values from FreePIE to UDK? Maybe with the memory mapped file in windows?

You own a 3D printer? I envy you!

Post by **BadKarma** » Mon May 21, 2012 4:09 am

CyberVillain wrote:Nice that it works better. Since you can set the pitch and yaw from code isnt there wanyway to pass the values from FreePIE to UDK? Maybe with the memory mapped file in windows?

Most definitely and this can be done by binding a DLL file to UDK. But since time is of the essence and DLL coding is not my strong point I am forced to do it this way. I will however continue to work on this project once it's officially done. Simply because it would be an awesome (and relatively cheap) way for others to use the Razor and FreePIE as a headtracker in UDK. Perhaps if you're willing and have the time to spare, it might be interesting for you to integrate this with UDK and it's DLL binding functions?

CyberVillain wrote:You own a 3D printer? I envy you!

I wish! I just have real easy access to one. Our school invested in one

Won't be able to post any pictures today though as the guy who runs the shop and printer didn't have time this morning. Will keep you posted on this though.

By the way, did you know FreePIE and the Razor IMU plugin just saved me from redoing my semester?

I put so much effort in DLL coding research, but almost gave up until I came across the Mean to be Seen forum, and specifically a couple of your threads. So again, THANK YOU.

Post by **CyberVillain** » Mon May 21, 2012 5:02 am

Sorry, dont have the time to learn their engine, but if you can import any kind of dll you could use the Freetrack client dll and send Freetrack commands from FreePIE.

Glad I could help, where can I send the consultant invoice?

Post by **BadKarma** » Mon May 21, 2012 7:20 am

Haha, well, you can count on a donation when all this is over, that's for sure!

Post by **brantlew** » Fri Jun 01, 2012 8:53 am

Wiimote Button Support

I added basic button support for multiple wiimotes. The wiimotes must be connected via blue-tooth before running a script. Blue-tooth connection is tricky. First you must click both 1 and 2 buttons on wiimote until the LED's blink. Then go through the Windows blue-tooth connection wizards (no pass-code) and wait until you see a taskbar popup that says the device has been successfully connected. Then you must quickly run your script the first time to connect the wiimotes. If you don't complete all this before the Wiimotes stop blinking, they will timeout and you will have to start over again. Often I would frustratingly just connect the wiimotes using GlovePIE first because it gracefully handles wiimote registration. Once the devices are registered properly however, you can start, stop, and restart FreePIE scripts without problems.

FreePIE will handle up to 4 wiimotes and light up the LEDs to indicate the numbering order. Within the script, you must indicate the wiimote index number (even if you are just using 1 wiimote). The syntax is wiimote[n]:getValue() (ie: wiimote[0].getUp() ) Currently only buttons are supported and here are the available functions.

getUp()
getDown()
getRight()
getLeft()
getA()
getB()
getMinus()
getHome()
getPlus()
getOne()
getTwo()

Here is a sample script for dual-hand Wiimote control of a FPS game

Code: Select all

MOUSE_SPEED = 3;

-- Right Wiimote
if (wiimote[0]:getRight()) then
   mouse:setDeltaX(MOUSE_SPEED);
elseif (wiimote[0]:getLeft()) then
   mouse:setDeltaX(-MOUSE_SPEED);
end

if (wiimote[0]:getUp()) then
   mouse:setDeltaY(-MOUSE_SPEED);
elseif (wiimote[0]:getDown()) then
   mouse:setDeltaY(MOUSE_SPEED);
end

-- Left Wiimote
if (wiimote[1]:getUp()) then
   keyboard:setKeyDown(Key.W);
else
   keyboard:setKeyUp(Key.W);
end

if (wiimote[1]:getLeft()) then
   keyboard:setKeyDown(Key.A);
else
   keyboard:setKeyUp(Key.A);
end

if (wiimote[1]:getDown()) then
   keyboard:setKeyDown(Key.S);
else
   keyboard:setKeyUp(Key.S);
end

if (wiimote[1]:getRight()) then
   keyboard:setKeyDown(Key.D);
else
   keyboard:setKeyUp(Key.D);
end

Post by **CyberVillain** » Sat Jun 02, 2012 5:48 am

Nice work! I removed the reference to Wiimote from GUI and made a script that copies it to the correct folder. Is it a bug with the Wiimote lib or whats the problem with realease?

Post by **brantlew** » Sat Jun 02, 2012 7:36 am

I'm not sure. It works in my Console release and debug fine, but it doesn't work in my GUI release. Maybe it's just a compilation error. I will retry it again later.

Post by **CyberVillain** » Sun Jun 03, 2012 6:14 am

I can try it out later tonight, what I did when i tried out indexed plugins were to declare an indexer for the global plugin, dont think I tried it in release though. I will try to find some time later tonight to try it out

Post by **brantlew** » Sun Jun 03, 2012 9:29 am

I have been putting together a script for dual-hand wiimote control of SkyRim. This is the first time I've really done any integration and real-world testing of the project, and I have uncovered a number of lingering bugs and usability issues. I uploaded fixes to several modules. The previous script can now be written simpler like this...

Code: Select all

MOUSE_SPEED = 3;

-- Right Wiimote
if (wiimote[0]:getRight()) then
   mouse:setDeltaX(MOUSE_SPEED);
elseif (wiimote[0]:getLeft()) then
   mouse:setDeltaX(-MOUSE_SPEED);
end

if (wiimote[0]:getUp()) then
   mouse:setDeltaY(-MOUSE_SPEED);
elseif (wiimote[0]:getDown()) then
   mouse:setDeltaY(MOUSE_SPEED);
end

-- Left Wiimote
keyboard:setKey(Key.W, wiimote[1].getUp());
keyboard:setKey(Key.A, wiimote[1].getLeft());
keyboard:setKey(Key.S, wiimote[1].getUp());
keyboard:setKey(Key.D, wiimote[1].getRight());

I'll publish the full SkyRim script in a few days designed for untethered back-top usage - complete with Vuzix head-tracking and dual wiimote control.

Post by **CyberVillain** » Sun Jun 03, 2012 10:58 am

It worked in release for me, I removed all Wiimote code since I dont have a wii mote. But the indexer part of your code works.

Make sure to "Clean solution" every time when you build the GUI since it does not have direct references to project, I've noticed that VS does not allways copy the dll to the plugin folder.

Post by **brantlew** » Sun Jun 03, 2012 12:02 pm

ok thanks for checking it out

Post by **CyberVillain** » Mon Jun 04, 2012 12:28 pm

I'm thinking of moving the Code to Github, this way people can contribute code without being part of the team (With the pull request mechanism). We can keep the Project place at Source forge..

What do you say?

Post by **brantlew** » Mon Jun 04, 2012 12:42 pm

I've never used Git so I don't know much about the technical details, but I have heard it described by Linus Torvalds. Right now I think there are only two developers working on FreePIE so organizationally, I don't think it matters one way or the other. However there are some benefits to using Git's uncontrolled source model. For example, I would like to create a console-only version of FreePIE, but with the SourceForge model I was afraid that it would lead to developer conflict. With Git however, I would just be creating another branch.

Post by **CyberVillain** » Mon Jun 04, 2012 1:09 pm

brantlew wrote:I've never used Git so I don't know much about the technical details, but I have heard it described by Linus Torvalds. Right now I think there are only two developers working on FreePIE so organizationally, I don't think it matters one way or the other. However there are some benefits to using Git's uncontrolled source model. For example, I would like to create a console-only version of FreePIE, but with the SourceForge model I was afraid that it would lead to developer conflict. With Git however, I would just be creating another branch.

If we moved to Git anyone can make a branch, I'm sure there are lurkers that would contribute if it was easy todo, its not currently easy with Sorceforge / SVN. I've also got requests in t he form of MTBS PMs to move the code to Git

We could ask Neil to set up a sub forum Dedicated to FreePIE and close down the Source forge site completely.

Post by **CyberVillain** » Sat Jun 09, 2012 4:26 pm

Project page moved to Github

https://github.com/AndersMalmgren/FreePIE

Post by **mxxmmxxm** » Sun Jun 10, 2012 8:42 pm

Has anyone tested the Wiimote button driver with a Wiimote Plus yet (The one that calls itself RVL-CNT-01-TR, that started coming out last winter)?

GlovePIE has recently decided to stop working entirely on my system, and given the.... difficulties of its creator and Vuzix hardware, I'd love to switch over if it's worth setting up a build environment locally.

Post by **brantlew** » Sun Jun 10, 2012 10:05 pm

I have some old Wiimotes from the first run of the Wii and also some Wiimote Pluses that I bought about 6 months ago, and they both work fine. Only button support right now though. No IR camera or motion. FreePIE pretty much came abouty as a response to GlovePIE incompatibility with Vuzix, and that is one of the best uses for it. Let me know if you need scripts for Vuzix and Wiimote integration.

Post by **CyberVillain** » Mon Jun 11, 2012 12:39 am

You do not need dev env if you just want to use FreePIE normally, the current binary that can be downloaded does not have WiiMote support though, I was hoping to have time to add the Intellisense feature before next binary release, but maybe I have to release a new binary before that.

Post by **CyberVillain** » Mon Jun 11, 2012 12:46 am

btw Brantlew, let me know if you have any problems getting Git to work. I'm also using MSBuild for the new build process (To extract Git revisionnumber into the assembly info etc) it looks like MSBuild should work with express, let me know if it doesnt

Post by **Neil** » Mon Jun 11, 2012 3:25 pm

Just wanted to add that now you have a dedicated forum, there is no need to have a run-on thread. You can break things up now!

Regards,
Neil

Post by **profvr** » Mon Jul 02, 2012 5:12 pm

Hi, this is an interesting project but have you looked at existing tracker drivers libraries that are out there? Whilst it doesn't deal with the output emulation, I would look at Virtual Reality Peripheral Network (VRPN) from University North Carolina Chapel-Hill for the input side. Its public domain and supports practically every tracker out there including Freespace, Wiimote, etc.; most companies developing trackers release software for it or someone has added it at some point (VRPN has a very long history!). Another nice feature is you can run your tracker over the network (something we do a lot of in our lab at least for testing; the trackers are plugged in to well-known machines where the drivers run very reliably, and client software connects over the network ).

Whilst adding specific driver support to FreePIE is probably quite easy, and dealing with VRPN initially would be tougher, you would get access to a lot of devices and a lot of configuration options for them.

There would still be lots to do in FreePIE, such as calibrating coordinate systems, filtering, range detection, etc.

I am looking to see if we can release some of our locally tracker filtering code, it will deal well with the head-tracked scenario. We do have a freespace tracker already and have used it in HMD demos, butwe have lots of trackers in our lab so we tend to use trackers with absolute position tracking for HMD demos, so haven't focused on tracker filtering specifically for the freespace (until now).

Post by **brantlew** » Mon Jul 02, 2012 6:34 pm

We talked a bit about implementing this as a VRPN node in the beginning. I remember thinking that the VRPN interface seemed to have a lot of extra baggage, I was concerned about the network overhead, and frankly - the code looked a bit too academic. But there were no real plans made or design spec'd out. It just sort of took off and evolved on its own as open source projects do. There are still long range plans to implement a VRPN interface into the utility. There is nothing stopping it from acting as a VRPN node. Personally I think it should preserve its kernel as a tight local interface. Actually even tighter than it is now - ideally as a linkable C dll that would allow third party apps to link directly to it without compilation or network connections.

Post by **CyberVillain** » Tue Jul 03, 2012 2:00 am

Hi. profvr
We decided that we could create a VRPN plugin down the road and jack that into FreePIE like any other I/O plugin since we still needed emulation.
If you would like to do it I would gladly pull that into the Master branch on Github

Regards, Anders

edit: That filtering stuff sounds neat, we really need better filters ,I just did a quick and dirty one to have something

Post by **profvr** » Tue Jul 03, 2012 2:09 pm

brantlew wrote:We talked a bit about implementing this as a VRPN node in the beginning. I remember thinking that the VRPN interface seemed to have a lot of extra baggage, I was concerned about the network overhead, and frankly - the code looked a bit too academic. But there were no real plans made or design spec'd out. It just sort of took off and evolved on its own as open source projects do. There are still long range plans to implement a VRPN interface into the utility. There is nothing stopping it from acting as a VRPN node. Personally I think it should preserve its kernel as a tight local interface. Actually even tighter than it is now - ideally as a linkable C dll that would allow third party apps to link directly to it without compilation or network connections.

No argument about it not having some complexity as it does a lot of different things beyond just reading devices (logging and replay is one I've used a lot to debug). However there are good features in there which are valuable downstream. E.G. I think I recall you can run client and server in one process, which has the effect of just cleanly separating off the device thread and should have no latency overhead on a single machine.

Post by **CyberVillain** » Thu Jul 12, 2012 6:07 pm

Uploaded new binary, lots of changes, code completion, Hillcrest Freespace support, Property syntax now supported etc!