TrackIR emulation coming soon

[CyberVillain]

My brother and I have made huge progess on emulating the TrackIR, we will have a plugin ready in the next couple of days.
Must give all credit to my brother who unlike me is a C++ pro and reverse engineered the NaturalPoint API.

We also want to support reading from the device, for this to work we need someone with a TrackIR 5 to test it out, are there anyone here with a TrackIR5?

Also,
there is a drawback with emulating the NaturalPoint API, they have included a copyrighted string that the games read and test against a hash, the string is classed as a poem (NaturalPoints are patent trolls) so if we include this string in the FreePIE source we will have NaturalPoints lawyers breathing down our necks in no time, there for we will leave this string out of the code and you will have to find it on the net and set it yourslef in the configuration window of FreePIE. This way we can get NaturalPoint emulation without breaking any Copyrights

[WiredEarp]

Awesome news CyberVillain, congratulations to you and your brother.
Unfortunately, I only have a TIR4.

[CyberVillain]

Awesome news CyberVillain, congratulations to you and your brother.
Unfortunately, I only have a TIR4.

Thanks! Do you know if there are any big changes in the protocol? I guess testing against v4 is better than nothing?

We will use the official dll when reading so it should work

[cybereality]

Very cool.

[2EyeGuy]

There are two difficulties with emulating TrackIR.

One is the supposedly copyright supposed "poem". NaturalPoint don't have a legal leg to stand on, but they will threaten everyone and the courts can't be trusted to use common sense.

The second is the encrypted protocol used for some of the newer games.

I get around both issues with some clever hacking, without any need for the user to enter the poem, and without modifying any program code or memory of the game.

The NaturalPoint SDK probably consists of the DLL file (which we replace), plus a standard static library or source code supplied by NaturalPoint which calls that DLL in a standard way. There are several versions of the NaturalPoint library, but generally they involve reading the registry to get the location of the DLL, loading the DLL, loading the functions from the DLL and storing the address of each in memory, storing the poem in an array on the stack, calling your DLL function with an address of a stack buffer to put the poem into, then comparing the two buffers. So usually in your function, you can copy the poem from a standard offset to the address of the buffer passed to your function.

The naturalpoint code for the encrypted games just has a function that calls your GetData function, then calls the decryption function on it afterwards. If you change the return address in your GetData function, you can skip over the decryption code. The amount you have to skip and the stack variables vary slightly from game to game though.

[CyberRascal]

2EyeGuy: Hey, thanks for chiming in. (I'm the alleged brother noted above)

It does sound good, reading the expected signature from the game itself. Can you tell me what the offset from the input buffer is?

Overwriting the return address seems a bit risky though - I'm not an expert on memory security, but what about canaries for example? I guess you can just use a compiler that doesn't generate canaries for the dll though.. And it's probably easy to overwrite the return address of a function we are compiling without overwriting the canary..

Also, keeping track of the offset for each game seems like it would require significant maintenance on the part of FreePIE... But maybe I'm wrong there too?

PS. Obviously the "reverse engineering" my brother credits me for is just compiling a DLL that exports the correct functions + returning that 'poem'. Nothing much, in other words

[ITGuy327]

I have a TrackIR 5. I'd be interested in helping. I'm a dev but for business apps, I'm getting into all 3d motion control gaming type stuff though, including NUI. Let me know if I can help ya out.

[WiredEarp]

Thanks! Do you know if there are any big changes in the protocol? I guess testing against v4 is better than nothing?

I'm pretty sure TIR4 and TIR5 have the same protocol etc, just TIR5 has a wider FOV I believe.

[CyberVillain]

Cool we will put up a binary here when you guys can test!

[ITGuy327]

I've been checking out the source code. If you have any code, I can compile and test. That is of course if you could brief me on how to get the other plugins available in the GUI, it was really late at night when I first looked

[CyberVillain]

I've been checking out the source code. If you have any code, I can compile and test. That is of course if you could brief me on how to get the other plugins available in the GUI, it was really late at night when I first looked

Its in my brothers fork, but I do not think he has committed the read code to it yet, will let you know when it is.

What do you mean by "how to get the other plugins available in the GUI"?

[ITGuy327]

Re: GUI Plugins - The menu only shows a few of the plugins (iPhone, Ipod,etc) but I don't see FreeTrack or some of the others that I see source files for.

[CyberVillain]

Re: GUI Plugins - The menu only shows a few of the plugins (iPhone, Ipod,etc) but I don't see FreeTrack or some of the others that I see source files for.

Thats very strange, not even with the precompiled binary from downloads? Also make sure to clean the GUI project if you are trying FreePIE from Visual Studio, it has problems understandfing that the Core.Plugin needs to be built (Its not directly referenced).

edit: ah, you mean from the menu? That is only plugins that need settings

test to write something in the code window, you will get code completion

[ITGuy327]

I didn't try the compiled binaries, but I'll check them out. Thanks for the note on VS, I'll look into that also.

[CyberVillain]

I didn't try the compiled binaries, but I'll check them out. Thanks for the note on VS, I'll look into that also.

see my edit above

[ITGuy327]

Nice! Thanks will do

[CyberVillain]

Ok, i've tried to fake TrackIR successfully in Grid Race Driver and also Future Pinball, if you have any TrackIR titles please try.
Check this site for your title and if its encrypted.
http://naturalpointofview.blogspot.se/p ... fixer.html

You guys with TrackIR hardware, please try and see if you can read trackir successfully, I've tried to output fake TrackIr from Freetrack > Freepie, and that works...

Put the plugin dll under the plugin folder and npclient.dll under FreePIE root

[ITGuy327]

Sorry guys, apparently I'm a little slower than I thought.

Can anyone post a sample TrackIR lua for testing. I got the following error.
"Reading not supported - real trackir dll not loaded."

Initially I didn't realize I needed to run a TrackIR game, so I ran FreeSpace 2 Open Source and Tracking from TrackIR worked and the diagnostic:watch is producing output. I might be on the right path, I think, would love additional input.

Code: Select all

keyboard:setKey(Key.W, trackIR.Pitch);
keyboard:setKey(Key.A, trackIR.Yaw);
diagnostics:watch(trackIR.Pitch);
diagnostics:watch(trackIR.Yaw);

Question: If the game supports TrackIR, what would the purpose of FreePIE be?
Question: If the game doesn't support TrackIR, can I use TrackIR hardware w/ FreePIE to emulate it via mouse?

Sorry for all the static, but this is a great project. Thanks

[CyberVillain]

Sorry guys, apparently I'm a little slower than I thought.

Can anyone post a sample TrackIR lua for testing. I got the following error.
"Reading not supported - real trackir dll not loaded."

My brother has to get back to you on that error, but since diagnostics:watch(trackIR.Yaw) works (?) it sounds like reading works. For reading this would do as a test

Code: Select all

diagnostics:watch(trackIR.Yaw);
diagnostics:watch(trackIR.Pitch);
diagnostics:watch(trackIR.Roll);

For writing (Which is not very useful for someone that owns a TrackIR, unless you want to alter the TrackIR signal before it reaches the game).If you read / write from trackIR its important that TrackIR runs before executing the FreePIE script, otherwise TrackIR will overwrite the dll paths in registry

A good example for trackIR > trackIR example is to put the signal through some kind of filter, like a curve (Define curves under settings in GUI).

Code: Select all

trackIR.Yaw = yawCurve:getY(trackIR.Yaw)

note: name yawCurve is defined in settings

Another usefull example could be generic IMU > trackIR (I dont rember if the ahrsIMU outputs in deg or rad, but if its in rad just use math.deg on the output (TrackIR uses degs)

Code: Select all

trackIR.Yaw = ahrsImu.Yaw
trackIR.Pitch = ahrsImu.Pitch
trackIR.Roll = ahrsImu.Roll

Initially I didn't realize I needed to run a TrackIR game, so I ran FreeSpace 2 Open Source and Tracking from TrackIR worked and the diagnostic:watch is producing output. I might be on the right path, I think, would love additional input.

Well to test FreePIE's ability to output TrackIR to a game a TrackIR enabled game is needed Thats what I meant

Code: Select all

keyboard:setKey(Key.W, trackIR.Pitch);
keyboard:setKey(Key.A, trackIR.Yaw);
diagnostics:watch(trackIR.Pitch);
diagnostics:watch(trackIR.Yaw);

You cant do it like that, keyboard:setKey(Key.W, trackIR.Pitch), what you can do though

Code: Select all

keyboard:setKey(Key.W, trackIR.Pitch > 45)

This will make the game character start to walk if you pitch 45 degs or more

Question: If the game supports TrackIR, what would the purpose of FreePIE be?

1) Support in TrackIR games for none TrackIR devices, sadly there arent many titles that support the open and Free Freetrack interface.

2) Alter the TrackIR signal before it its the Game, or to get better Mouse / Keyboard emulation than the build in support in TrackIR

Question: If the game doesn't support TrackIR, can I use TrackIR hardware w/ FreePIE to emulate it via mouse?

Yes if FreePIE can read from TrackIR (You need to confirm this ) then you can use FreePIE to output Freetrack, Mouse, Keyboard, virtual joystick etc.

Sorry for all the static, but this is a great project. Thanks

No problem

[CyberRascal]

Sorry guys, apparently I'm a little slower than I thought.
Can anyone post a sample TrackIR lua for testing. I got the following error.
"Reading not supported - real trackir dll not loaded."

It seems that FreePIE [thinks it] cannot find the real trackir dll from the registry. Could you tell me, with FreePIE shut down, what values exist in HKEY_CURRENT_USER\Software\NaturalPoint\NATURALPOINT\NPClient Location\?

Can you also run a script using TrackIR in FreePIE, and check again (with the script still runnning)?

In my all-compassing stupidity I disabled logging for the TrackIR module in the files you downloaded. I will add some more thorough logging to figure out what exactly is happening.

One thing that is a bit worrying is that there are so many different programs trying to tell the games to use their DLL - atleast TrackIR and FreePIE, and the order in which you start the programs determines the output. I'll have to consider what a clean way to deal with this is.

I (or CyberVillain) will upload a new set of DLL's in a while. If you can find the time, I'd appreciate it if you could test it again. I'll also post some test scripts I'd like you to try. Thanks for the effort!

[CyberVillain]

Brother, you do not require a NPClient in registry?

If you dont own a TrackIR or use Freetrack, GlovePIE you wont have that entry in the registry

[CyberRascal]

A real TrackIR dll (that is, one that isn't inserted by FreePIE) is only required when trying to read from TrackIR. I guess I could default to reading from the fake one, but that seems a bit weird.

It might be clearer to partition it into two different plugins / entities: FreepieTrackIR and RealTrackIR or something like that.. One is write and readable (write to fake, read from fake) while the other is only readable (reads from the "real" TrackIR dll).

This is what GlovePIE does I think.. It grates a little to come to the same conclusion

What do you think?

[CyberVillain]

I like one with write / read property like it is now, GlovePIE's syntax is crap

But "Reading not supported - real trackir dll not loaded." cant be thrown if you try to write, thats all

[CyberRascal]

Alright. If anyone else has any future or current opinions on how the plugin should work though, please speak up

And no, that error should only occur when reading. It only occurs in one execution path, which is right before executing the read function from the real DLL. The question is though, why it didn't find the real DLL when apparently it existed.

I'll add some more logging...

[CyberVillain]

Alright. If anyone else has any future or current opinions on how the plugin should work though, please speak up

And no, that error should only occur when reading. It only occurs in one execution path, which is right before executing the read function from the real DLL. The question is though, why it didn't find the real DLL when apparently it existed.

I'll add some more logging...

Strange thing is that ITGuy327 reported that he got stuff in the diagnostics window of freepie.. ITGuy327?

[CyberRascal]

Yes, it's a little strange, but my guess is that it wasn't at the same time.

Anyway, a new dll is ready. This one logs to the plugin folder in a file named TrackIRLog.txt. Also a few other improvements. Be wary though, the log file can sometimes grow pretty fast. Same instructions for "install" as last time.

Thanks to anyone with or without trackir that can test these two scripts:

----------------------

The first one simply simulates yaw using the faking infrastructure. Start the script, start any TrackIR enabled game without encryption and see if you get the expected output (i.e yaw goes from -90 -> 90)!

if(starting)
then
x = 0
end

x = x + 0.01;

if(x > 90)
then
x = -90
end

trackIR.Yaw = x

------------------------------------

This one simply writes updates to yaw in the console everytime there is an update. Please pay attention to
1) whether it works at all
2) any weird behaviour

if(starting)
then
trackIR.Update:Add(function()
diagnostics:watch(trackIR.Yaw)
end)
end

------------------------------------

[ITGuy327]

Thank you for the info. I will test anything you give me and post back. I agree the error and FreePIE output was probably just a timing issue because i was consistently able to see trackir readings.

Any chance the code is up yet?

I will get back with any results i have. Thanks again to all.

[CyberRascal]

Thanks ITGuy, appreciate it

The latest (at the time of writing) DLL's are attached to my last post, the long one with the scripts and everything. That's how you got the last ones, right?

[ITGuy327]

I got the dll's. I was curious to see the C# code.

My results for both scripts were a crash of FreePIE. It also did not seem to like the if (starting) portion, so I tried without it and I got some output in the console but then FreePIE crashed.

It also seemed that the TrackIR didn't register the game loading when the script was running first. I'm also attaching my log file, which ended up in the FreePIE root folder as opposed to the plugin folder.

Please let me know if there is more that I can do.

[CyberRascal]

Thanks, much appreciated! Sorry I misunderstood you. Heh. The code is at https://github.com/maxmalmgren/FreePIE though the unmanaged NPClient code is not available as of yet. The all in Core.Plugins\TrackIRPlugin.cs and Core.Plugins\TrackIR\

At moments like this, I wish I could attach a debugger or access the unmanaged stacktrace. Does FreePIE crash immediately or just hang?

If you are running a script with a statement like trackIR.Yaw = 10, i.e a script that attempts to set some heading data, the freepie NPClient.dll is temporarily injected until you stop the script (or freepie crashes, and you start and stop the script again - since it crashes immediately and freepie doesn't catch an exception it's probably a segmentation error or something). This means that any game will load the FreePIE dll and not the TrackIR one.

The weird thing is that the crash seems to occur in the real DLL. It seems like this is a part where you restarted the script many times due to crashes:

Code: Select all

Found real trackir dll at: C:\Program Files (x86)\NaturalPoint\TrackIR5\
Loading dll at C:\Program Files (x86)\NaturalPoint\TrackIR5\NPClient.dll
Found real trackir dll at: C:\Program Files (x86)\NaturalPoint\TrackIR5\
Loading dll at C:\Program Files (x86)\NaturalPoint\TrackIR5\NPClient.dll
Found real trackir dll at: C:\Program Files (x86)\NaturalPoint\TrackIR5\
Loading dll at C:\Program Files (x86)\NaturalPoint\TrackIR5\NPClient.dll
Found real trackir dll at: C:\Program Files (x86)\NaturalPoint\TrackIR5\
Loading dll at C:\Program Files (x86)\NaturalPoint\TrackIR5\NPClient.dll
Found real trackir dll at: C:\Program Files (x86)\NaturalPoint\TrackIR5\

Does the script below crash freepie? Do you get a FreePIE error on the if(starting) line or a straight up crash? It should give you output in the "watch window", not the console.. But maybe that's what you meant?

It's possible for FreePIE to behave weirdly if you execute the below code without the if(starting). That means that you add a listener to the event trackIR.Update a hell of a lot times per second, which means that the method gets called very often after a few seconds, can make freepie behave weirdly.

Code: Select all

if(starting)
then
    trackIR.Update:Add(function()
        diagnostics:watch(trackIR.Yaw)
    end)
end

Again, thanks for your time!

[ITGuy327]

Thanks for posting the code.

I have one dumb question about Git, since I'm more familiar with SVN, if I've cloned the FreePIE repo and want to keep it synchronized with the server, what is the command? I tried several but it was not getting the TrackIR code that you pointed to on the server. Thanks

FreePIE crashed pretty quickly. I will test out the additional script. Gotcha on the starting command and the excessive calls, it makes sense now.

Yes, I meant console since I didn't have any watch statements.

Have you considered using Log4Net, I've used it in several projects and it's made my life a little easier. If you're interested I could probably add it in. Perhaps you could at minimum write the Stacktrace when the error occurs, which is what I've done in some of my apps.

Thanks again.

[CyberVillain]

Thanks for posting the code.

I have one dumb question about Git, since I'm more familiar with SVN, if I've cloned the FreePIE repo and want to keep it synchronized with the server, what is the command? I tried several but it was not getting the TrackIR code that you pointed to on the server. Thanks

FreePIE crashed pretty quickly. I will test out the additional script. Gotcha on the starting command and the excessive calls, it makes sense now.

Yes, I meant console since I didn't have any watch statements.

Have you considered using Log4Net, I've used it in several projects and it's made my life a little easier. If you're interested I could probably add it in. Perhaps you could at minimum write the Stacktrace when the error occurs, which is what I've done in some of my apps.

Thanks again.

Log4net wouldnt help here, all the LUA execution paths are protected and exceptions thrown from plugin or script should end up in the Error window of FreePIE but some unmanaged crap in the NPClient makes the Managed code crash.

I use TortoiseGit http://code.google.com/p/tortoisegit/

Fork my brothers repo, go to https://github.com/maxmalmgren/FreePIE and press the Fork button on the right

Now right click windows explorer select Clone from TortoiseGit and enter the url to your forked repo

edit: To push changes to your Repo just use the Commit command from TortoiseGit and then followed by a Push.

[ITGuy327]

Thanks for the details on git. I understand about the logging now, thanks for the explanation. I will give this a go.

[mahler]

Thanks for posting the code.

I have one dumb question about Git, since I'm more familiar with SVN, if I've cloned the FreePIE repo and want to keep it synchronized with the server, what is the command? I tried several but it was not getting the TrackIR code that you pointed to on the server. Thanks

What you are probably looking for is:

Code: Select all

git pull

Git does need a little shift in mindset compared to CVS and SVN

Here is an interesting video Tech Talk: Linus Torvalds on git (to change your mindset)

If you've cloned the FreePIE, you can commit your changes to your own local machine. Then later you can upload those commits to the origin branch (git push)
However, if you don't have push-privileges, you can do something called a pull-request.

[ITGuy327]

Funny, I thought I tried a pull command but didn't see the new files. Anyways I likely just did it wrong.

Thanks, that's some great info, have to watch the video. It does seem a shift in mindset is required to a degree. Funny thing is I use TortoiseSVN and TortoiseHg and duh never considered there would be Tortoise for git. TortoiseGit did the trick for me. Thanks again

[CyberRascal]

I've got some updates for the DLL's. Didn't pay enough attention to the reading side since I can't test it myself other than from Freetrack, which is just a fake anyway.

Added: Call startup functions (might be this that caused the crash).
Added: Better error handling when calling functions.
Added: Logging of version and signature of the real DLL.

Please try these and see if anything works better Uploading the code for it in a minute... Will need cleanup when everything is stable though.

[CyberVillain]

Freetrack > Freepie works, and now Freetrack detects FreePIE as a client so something is different

[ITGuy327]

Freetrack detects FreePIE as a client

what do you mean by that?

just curious.

[CyberRascal]

There is a function called NP_RegisterProgramProfileID which is now called on start of a script using TrackIR. This is probably what in Freetrack triggers a popup for CyberVillain. It says something about a new client registered or so. This id is probably assigned by NaturalPoint or something, but all I do is feed the function a random number for now (but the same one each time).

[ITGuy327]

I receive the following error when I run the scripts.
"Error occured while reading from TrackIR dll: 7"