It is currently Thu Feb 21, 2019 6:25 pm



Reply to topic  [ 37 posts ] 
 Hillcrest/Rift tracker support 
Author Message
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
I got my Hillcrest FSRK-USB-2 tracker chip (the tracker used in the Oculus Rift) 2 days ago and started implementing it as a FreePIE module. It's been quite difficult so far because the Hillcrest documentation is sparse and unfriendly, the sample code has bugs in it, and C# has been fighting me every step of the way. But I think I'm finally past all the hard parts now and expect to have support completed by this weekend.

C# has proven to be a frustrating language choice for FreePIE where module development is concerned because all device API's are written and maintained in C and proxying function calls between .NET and C is awkward. There are sometimes managed wrapper libraries available, but they always seem to be one or two versions behind the most recent API spec - even managed Microsoft libraries. This has been a difficult issue for several modules, but for the Freespace SDK it has been the most difficult. libfreespace (the C version) is on version 6 now but libfreespace.NET is 3 years old and is useless. After several failed attempts to interface the tracker from C#, I just decided to implement the tracker interface in C (via libfreespace 6) and then write a proxy DLL to shuttle the values to C#. A messy solution but one that saves me a lot of hassle.

So hopefully I will have a working module soon.


Thu Jul 05, 2012 12:33 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
Cool, how good is the trakcer?

Code completion is soon complete, i'll release a new binary soon, it would be really cool if we could get axis support for the WiiMote since thats the mainstream device.

_________________
FreePIE
My blog


Thu Jul 05, 2012 2:04 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
Yeah its a bit of hassle to invoke C code from managed code, but if we wrote the program in C we would have needed to use the win32 api for the GUI and thats an even bigger PAIN :D

Easiest way is either to build a managed C++ project and wrap the SDK, or write a unmanaged static C++ library and wrap the SDK.

Calling static C++ methods (As long as they return simple types) is really easy in .NET

_________________
FreePIE
My blog


Thu Jul 05, 2012 2:40 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
The tracker is fine, but nothing special..equivalent to other trackers I have used. The best thing about it is the ease of use. Since the board has an integrated USB and the interface library is mature and already loaded, you literally just plug it in and it's ready to go.

CyberVillain wrote:
Calling static C++ methods (As long as they return simple types) is really easy in .NET


Yeah that's basically what I'm doing now. The libfreespace function parameters are complex. The functions return
union {
struct;
struct;
...
}

objects so difficult to map easily to C#. So I'm just wrapping all that with easy functions that return a few floats instead.


Thu Jul 05, 2012 6:19 am
Profile
3D Angel Eyes (Moderator)
User avatar

Joined: Sat Apr 12, 2008 8:18 pm
Posts: 11376
Reply with quote
Cool. I ordered one of these trackers the other night. I'm hoping its at least as good as the one on the Vuzix, I was happy with that.

_________________
check my blog - cybereality.com


Thu Jul 05, 2012 6:44 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
@cyber: Here, I'll save you a couple hours of debugging. The example code on the website has bugs in it for the FSRK-USB-2 and none of the compiled examples will work either (except MotionStudio). The bug is all over the code and looks like this...

Code:
if (freespace_isNewDevice(device) == FREESPACE_SUCCESS) { 
}


should be changed to...

Code:
if (freespace_isNewDevice(device)) {
}


Thu Jul 05, 2012 6:51 pm
Profile
3D Angel Eyes (Moderator)
User avatar

Joined: Sat Apr 12, 2008 8:18 pm
Posts: 11376
Reply with quote
Thanks for the heads up.

_________________
check my blog - cybereality.com


Thu Jul 05, 2012 7:42 pm
Profile
Golden Eyed Wiseman! (or woman!)

Joined: Fri Jul 08, 2011 11:47 pm
Posts: 1498
Reply with quote
Can't you just define FREESPACE_SUCCESS as true and save altering the code in all the different places?
Or is it used elsewhere for an actual purpose besides as a 'true'...


Thu Jul 05, 2012 8:59 pm
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
Yeah that's basically what I'm doing now. The libfreespace function parameters are complex. The functions return
union {
struct;
struct;
...
}

objects so difficult to map easily to C#. So I'm just wrapping all that with easy functions that return a few floats instead.


As long as its structs (Its a value type) it shouldn't be a problem, its classes that's a pain, and a union is just another struct if i recall correctly

_________________
FreePIE
My blog


Fri Jul 06, 2012 1:14 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
Implemented support for the device and uploaded the code. I created a small C dll called piefreespace.dll that proxies data between the FreePI FreeSpace plugin and libfreespace. The C code is checked in as well, but I didn't upload a project or make-file with it, but it has very few dependencies. Developers interested in providing direct C++ support for the device may find that code useful. One thing that I would like to implement later is a way to programmatically recalibrate the sensor direction. No binary build available yet.

Here is the implementation in action.




And here is the script that I used. (@CyberVillain... I LOVE the new syntax now with the member accessors so thanks for that) :)

Code:
if (starting) then
   freespace.ContinuousYawMode = true;
end

yaw = math.deg(freespace.Yaw);
pitch = math.deg(freespace.Pitch);
roll = math.deg(freespace.Roll);

mouse.DeltaX = -filters:delta(yaw * 10);
mouse.DeltaY = filters:delta(pitch * 10);


Last edited by brantlew on Sun Jul 08, 2012 11:23 pm, edited 1 time in total.



Sun Jul 08, 2012 3:28 pm
Profile
Certif-Eyed!

Joined: Sun Mar 25, 2012 12:33 pm
Posts: 660
Reply with quote
This is amazing news! With Emerson's Biclops, the RIFT will be almost fully compatible with most games right off the bat!
This will help to thrust it out into the common video-gaming space.

I wonder if Palmer will be getting more than he bargained for? ;)


Sun Jul 08, 2012 9:45 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
@zalo: Well if what Palmer says comes to fruition, we may be having a whole lot of native game support coming down the pipe. Still, there's a lot of room for hacking to get all the legacy games to work. Not sure if FreePIE meshes exactly with what Emerson & cyber are doing (I think they are going for a direct interface) but it doesn't hurt to have more options.


Sun Jul 08, 2012 11:33 pm
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
And here is the script that I used. (@CyberVillain... I LOVE the new syntax now with the member accessors so thanks for that) :)


Lua devs will be confused, but who cares? :D

_________________
FreePIE
My blog


Mon Jul 09, 2012 12:55 pm
Profile
One Eyed Hopeful

Joined: Tue Jun 19, 2012 8:34 pm
Posts: 9
Reply with quote
I received my FSRK-USB-2 and started hacking with it. It seems the linear Position fields are all set to 0 because the acceleration sensors are not good enough:

Have you tried to implement your own position tracking with a filtering method ?


Sun Jul 29, 2012 12:24 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
You can get linear acceleration values from the sensor. Position values cannot be determined accurately. Theoretically you can estimate the integral of the acceleration (sum the area under the acceleration curve) to get velocity and then integrate the velocity to get position. But it just doesn't work because the sensor errors are so large and the double-integration just amplifies that error. At best your position values will be off by several centimeters in just a couple of seconds - and often it will drift meters per second!!!! This is true of all affordable MEMS accelerometers - not just the Hillcrest.


Sun Jul 29, 2012 2:36 pm
Profile
One Eyed Hopeful

Joined: Tue Jun 19, 2012 8:34 pm
Posts: 9
Reply with quote
Snap :( ! I read somewhere John Carmack mentioning the FSRK-USB-2 allowed to build a full matrix update and I assumed it would be a 4x4 with rotation and translation but it seems we can only build a 3x3 rotation accurately....I wonder how this is going to impact the occulus rift: "Yaw" and "Pitch" should be fine but human "roll" movement are also accompanied with translation since the movement starts at the base of the neck....I have never tried a VR Headset so maybe the brain won't notice....

A solution would be to hard code the head translation for the Yaw...


Sun Jul 29, 2012 3:19 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
Carmack put in some "fake" translations so that all three rotations model the shape of the head and neck. So for example, looking down is not just a rotation around the camera midpoint, but is a rotations around a modeled head originating at a neck. Same with yaw and roll. These translation animations are "canned" however and not derived from actual translations pulled from the sensor.


Sun Jul 29, 2012 3:40 pm
Profile
One Eyed Hopeful

Joined: Tue Jun 19, 2012 8:34 pm
Posts: 9
Reply with quote
brantlew wrote:
Carmack put in some "fake" translations so that all three rotations model the shape of the head and neck. So for example, looking down is not just a rotation around the camera midpoint, but is a rotations around a modeled head originating at a neck. Same with yaw and roll. These translation animations are "canned" however and not derived from actual translations pulled from the sensor.


Source ?


Sun Jul 29, 2012 4:32 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
I'm pretty sure he mentioned it on one or two of the E3 videos that circulated.


Sun Jul 29, 2012 4:34 pm
Profile
3D Angel Eyes (Moderator)
User avatar

Joined: Sat Apr 12, 2008 8:18 pm
Posts: 11376
Reply with quote
I think he mentions it in this interview (but it could be another, there were a bunch of them):
http://www.giantbomb.com/e3-2012-john-c ... w/17-6164/

_________________
check my blog - cybereality.com


Sun Jul 29, 2012 5:29 pm
Profile
Cross Eyed!

Joined: Mon Aug 06, 2012 9:28 am
Posts: 128
Reply with quote
brantlew wrote:
You can get linear acceleration values from the sensor. Position values cannot be determined accurately. Theoretically you can estimate the integral of the acceleration (sum the area under the acceleration curve) to get velocity and then integrate the velocity to get position. But it just doesn't work because the sensor errors are so large and the double-integration just amplifies that error. At best your position values will be off by several centimeters in just a couple of seconds - and often it will drift meters per second!!!! This is true of all affordable MEMS accelerometers - not just the Hillcrest.


What's the reliability on the acceleration values? As in, can it reliably tell between if the user is stationery and moving in one direction or another? (I'd imagine that the drift would create a small degree of acceleration even when stable - but as long as it reports a reliable difference between stationary and moving, then you'd be able to discard the smaller drifting values.

If the reliability is good enough, wouldn't it be possible to use that data to initiate the translation movement in the view - while integrating slower camera based (i.e. kinect) absolute positioning information, to allow for some degree of view translation that should be adequate (i.e. better than nothing) for now?

The way I figure it - the accelerometers should help start the vector of translation at less than 20ms - the next 50ms, the view point accelerates at any arbitrary rate (determined by the average movement speed of a person), at the 70ms mark (a number I pulled from Carmack's mentioning about attempting to use the kinect (may have heard him wrongly)), the data from the kinect kicks in, overriding the assumed vector intiated by the accelerometers.

Of course there'd have to be some degree of prediction to smooth out the transition from accelerometer data to camera positional data - otherwise the view point would jump back 50ms to the kinect starting. But then doing that would probably add a few dozen more milliseconds as the system attempts to match the assumed trajectory up with the predicted, extrapolated data from the kinect.

But when head and torso movements regularly range into the second+ range, the jankiness of a couple hundred milliseconds may be an adequate compromise to having translation tracking at all. Additionally, there shouldn't be much in the way of overshoot - as you'd use the accelerometer data for stopping to halt the view point immediately, rather than continuing to use the lagging kinect data.

What do you think guys? Am I on the right track here?


Tue Aug 07, 2012 8:07 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
I think it's instructive to view what raw accelerometer readings actually look like. Here is a short sample I took at 60Hz, holding an accelerometer as stable as I possibly can and then slowly moving it forward for about 1 meter. You can see the motion along the green axis starting from about 2 seconds to 5 seconds.

Attachment:
RawAccel.jpg


This is what we mean when we are talking about IMU sensor noise. And this is a very constrained example. Things get much more chaotic as you start turning and moving at normal speeds. Can you see the motion signal - yes. Is it easy for the software to pick it out - sometimes. But it's rarely as easy as a simple cutoff and you generally need to smooth the signal to increase accuracy which always introduces delay. There is always a fine line that you traverse between accuracy and timeliness. You can have high accuracy and low responsiveness or you can have low accuracy and high responsiveness. Getting both is difficult. Complex mathematics and multi-sensor fusions definitely help but no one solution has really been the magic bullet to solve this problem in the general case with high accuracy.


You do not have the required permissions to view the files attached to this post.


Tue Aug 07, 2012 8:53 am
Profile
Cross Eyed!

Joined: Mon Aug 06, 2012 9:28 am
Posts: 128
Reply with quote
brantlew wrote:
I think it's instructive to view what raw accelerometer readings actually look like. Here is a short sample I took at 60Hz, holding an accelerometer as stable as I possibly can and then slowly moving it forward for about 1 meter. You can see the motion along the green axis starting from about 2 seconds to 5 seconds.

Attachment:
RawAccel.jpg


This is what we mean when we are talking about IMU sensor noise. And this is a very constrained example. Things get much more chaotic as you start turning and moving at normal speeds. Can you see the motion signal - yes. Is it easy for the software to pick it out - sometimes. But it's rarely as easy as a simple cutoff and you generally need to smooth the signal to increase accuracy which always introduces delay. There is always a fine line that you traverse between accuracy and timeliness. You can have high accuracy and low responsiveness or you can have low accuracy and high responsiveness. Getting both is difficult. Complex mathematics and multi-sensor fusions definitely help but no one solution has really been the magic bullet to solve this problem in the general case with high accuracy.


Thanks for the response. I understand quite clearly now the issue - by the time the acceleration signal overcomes the natural noisiness of the data, you've already been in motion for a few hundred millisecond - rendering this on-off initiation unusable.


Tue Aug 07, 2012 9:46 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
I wonder if the Freespace lib will be library of choice for Rift games, if so we should work on emulating that lib from FreePIE so any generic tracker can be used, for example from Cyber's Rift driver etc

_________________
FreePIE
My blog


Tue Oct 02, 2012 7:00 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
The Oculus SDK will be the library of choice and we should offer a plugin as soon as the SDK comes available.


Tue Oct 02, 2012 7:22 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
The Oculus SDK will be the library of choice and we should offer a plugin as soon as the SDK comes available.


I've totally missed that there will be a Official SDK, who will be doing it? As I understand it Palmer isn't a developer?

_________________
FreePIE
My blog


Tue Oct 02, 2012 7:33 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
Oculus has 10 or 15 employees now. I think the ScaleForm guys that joined Oculus have a big hand in the SDK development.


Tue Oct 02, 2012 7:56 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
Oculus has 10 or 15 employees now. I think the ScaleForm guys that joined Oculus have a big hand in the SDK development.


I have to read up in the RIFT forum I hear :D

_________________
FreePIE
My blog


Tue Oct 02, 2012 7:58 am
Profile
Two Eyed Hopeful
User avatar

Joined: Wed Oct 17, 2012 1:10 pm
Posts: 55
Location: Philadelphia, USA
Reply with quote
Does anyone have any advice for getting the FSRK USB ver 2 to work with FreePIE? I'm not having any luck. I used GlovePie before to map the mouse provided by the USB module to TrackIR so games will work with 2DOF, but I can't get the script you used to work. It'd be nice to get pitch/yaw/roll in games that support TrackIR.

Maybe I need to set my device to a different mode, don't know...

Thanks

Edit: I did get the Y/P/R values in the latest freepie and mapped to mouse (!) so I think this is just a trackir emulation problem.


Sat Dec 08, 2012 6:23 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
FreePIE recently changed their script format from LUA to Python scripting so many of the scripts will not work any longer. It should be pretty simple to change the syntax. If you look at CyberVillain's scripts in this thread you can get a flavor for the new syntax.

http://www.mtbs3d.com/phpBB/viewtopic.php?f=139&t=15052#p74672


Sat Dec 08, 2012 8:48 pm
Profile
Two Eyed Hopeful
User avatar

Joined: Wed Oct 17, 2012 1:10 pm
Posts: 55
Location: Philadelphia, USA
Reply with quote
Thanks - I figured that out eventually :) I added diagnostics.watch() and was able to see the freespace.* values coming from my FSRK-USB-2/FSM-6.


Sun Dec 09, 2012 2:41 am
Profile
Two Eyed Hopeful
User avatar

Joined: Wed Oct 17, 2012 1:10 pm
Posts: 55
Location: Philadelphia, USA
Reply with quote
It would be nice to get support built for the 250hz firmware (or just doing XYZ with custom fusion). Maybe just a toggle switch to use the Body Frame or User Frame for XYZ calculation. It still works right now with FreePie but since no fusion is being done it's not smooth.

Something like viewtopic.php?f=140&t=15247&start=1035#p90637

I am trying myself with a couple fusion algorithms (like the one shipped with FreePIE) but can't seem to get values in range.


Fri Dec 28, 2012 8:46 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
I do think an open source fusion DLL for the Hillcrest is a good idea if tmek wants to donate the code, but I believe something like that should be handled at the "driver" level before it reaches FreePIE. Not necessarily a kernel level driver, but at least in dedicated user space code - like a DLL built on top of libfreespace that just handles fusion and exposes the values. I think there are too many unpredictable timing pathways in FreePIE to guarantee a solid fusion algorithm within the framework since a single thread is used to run the Python script and interact with all the devices. So you can't predict the duty cycle because you don't know what sort of Python loops or blocking operations that thread might encounter on its way back around to your plugin.


Fri Dec 28, 2012 9:11 am
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
I do think an open source fusion DLL for the Hillcrest is a good idea if tmek wants to donate the code, but I believe something like that should be handled at the "driver" level before it reaches FreePIE. Not necessarily a kernel level driver, but at least in dedicated user space code - like a DLL built on top of libfreespace that just handles fusion and exposes the values. I think there are too many unpredictable timing pathways in FreePIE to guarantee a solid fusion algorithm within the framework since a single thread is used to run the Python script and interact with all the devices. So you can't predict the duty cycle because you don't know what sort of Python loops or blocking operations that thread might encounter on its way back around to your plugin.


You could handle it in the plugin thread (Have the plugin return a action from the Start method)

_________________
FreePIE
My blog


Fri Dec 28, 2012 11:20 am
Profile
Petrif-Eyed
User avatar

Joined: Sat Sep 17, 2011 9:23 pm
Posts: 2218
Location: Menlo Park, CA
Reply with quote
CyberVillain wrote:
You could handle it in the plugin thread (Have the plugin return a action from the Start method)


Sure, but there's not much benefit from running it in C#. Code samples for fusion and hardware interface libraries are always written in C, so porting it to C# and marshalling all of the data structures is just an exercise in tedium with no clear benefit.


Fri Dec 28, 2012 12:04 pm
Profile
Petrif-Eyed

Joined: Mon Jun 22, 2009 8:36 am
Posts: 2160
Location: Stockholm, Sweden
Reply with quote
brantlew wrote:
CyberVillain wrote:
You could handle it in the plugin thread (Have the plugin return a action from the Start method)


Sure, but there's not much benefit from running it in C#. Code samples for fusion and hardware interface libraries are always written in C, so porting it to C# and marshalling all of the data structures is just an exercise in tedium with no clear benefit.


Just giving all the options, but if you can do it in the driver thats fine

_________________
FreePIE
My blog


Fri Dec 28, 2012 1:22 pm
Profile
Petrif-Eyed
User avatar

Joined: Sat Jan 09, 2010 2:06 pm
Posts: 2255
Location: Perpignan, France
Reply with quote
brantlew wrote:
I think there are too many unpredictable timing pathways in FreePIE to guarantee a solid fusion algorithm within the framework since a single thread is used to run the Python script and interact with all the devices. So you can't predict the duty cycle because you don't know what sort of Python loops or blocking operations that thread might encounter on its way back around to your plugin.
The PS Move API uses the fusion code from Sebastian Madgwick and it can work at varying FPS with the time period being feeded to the algorithm at each refresh. Maybe that would be a good addition to FreePIE.

brantlew wrote:
Code samples for fusion and hardware interface libraries are always written in C, so porting it to C# and marshalling all of the data structures is just an exercise in tedium with no clear benefit.
The fusion code from Sebastian Madgwick which looks like state of the art at this time (ie. better than Kalman filters) is also available with an open source C# implementation :
http://www.x-io.co.uk/open-source-imu-a ... lgorithms/

Also this algorithm is able to do good fusion even at slow frequencies, in his paper the author said that the quality was equivalent at 50Hz and 500Hz, so the 250Hz firmware for the Hillcrest may not be needed.

The paper : http://www.x-io.co.uk/res/doc/madgwick_ ... report.pdf


Fri Dec 28, 2012 2:16 pm
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 37 posts ] 

Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group
Designed by STSoftware.