Very good IMU based skeletal tracking

[brantlew]

I found this very intriguing. It's another group that is doing an IMU based mo-cap suit - similar to the YEI video that went around but seemingly of a much higher quality. In particular, I thought the YEI video did a poor job of modeling walking. I don't think there was a physics model - only a kinematic skeleton controlled by sensors but when the guy walked around it looked like the feet were "sticky" and the model didn't work very well when the feet were not grounded. From what I can see in this video, these guys have a very good physics model and can handle all sorts of leaping, jumping, and running. It looks close to optical quality - but without the complexity and occlusion problems of optical. Very impressive stuff !

http://www.youtube.com/watch?v=DPGEG8U7Be8

http://www.youtube.com/watch?v=Gul7LWna3KI

[Chriky]

Yes that is very impressive, the drift looks well below perceivable levels, so unless you strictly need real world positions (e.g. for touchable environments) this looks like a totally viable solution.

I made a Hydra foot-tracking demo a while ago and put in a crude gravity simulation so it could do jumping; I imagine they are doing something similar.

There are a few things it will never be able to do such sliding, or swinging/climbing on something but those are very rare.

Any idea what "affordable" means here?

[brantlew]

Any idea what "affordable" means here?

No, but it is probably a relative term. Affordable for mo-cap, but not necessarily affordable for the average consumer. If you have access to raw parts and can build sensors, then your BOM on something like this is probably ~$100, but if you wanted to build this using ready sensor packages then I imagine you are looking at something in the $1000 - $2000 range.

[blazespinnaker]

That's awesome.

[MSat]

Very nice! Looks pretty darn robust (though with quite a bit of latency?). Reminds me of Xsens' system. The biggest downside is the number of required sensor nodes, which even if costs could be substantially reduced, makes it less than ideal for most consumers.

[brantlew]

Hard to tell how much latency there is even if the videos are perfectly synchronized. My guess is it falls somewhere between Kinect and IR LED latency - a pretty good experience. I keep thinking how awesome this would be for large scale environments. A very nice tradeoff of performance and scale.

Edit: I have been convinced for a good while that a sensor suit is probably the most general solution that could cover both ODT and free-walk scenarios and work despite any occluding clothing, harnesses, handheld devices, or terrain props.

[Dantesinferno]

I am going to have some very good news for you @brantlew ... Give me a few months, probably near the end of August and you will see!

[MSat]

Hard to tell how much latency there is even if the videos are perfectly synchronized. My guess is it falls somewhere between Kinect and IR LED latency - a pretty good experience. I keep thinking how awesome this would be for large scale environments. A very nice tradeoff of performance and scale.

Edit: I have been convinced for a good while that a sensor suit is probably the most general solution that could cover both ODT and free-walk scenarios and work despite any occluding clothing, harnesses, handheld devices, or terrain props.

If this particular product is mainly intended for mocap and not realtime use, then I suppose latency isn't a concern. I wouldn't be surprised if an multi-node IMU-based system could have low latency. After all, the Rift's headtracker is quite good.

I'm with you that this is probably the best solution for large environments particularly when limb tracking is required. However, you still might want another system for at least rough absolute positional tracking, as it's still technically possible to introduce drift in this system.

[brantlew]

If this particular product is mainly intended for mocap and not realtime use, then I suppose latency isn't a concern. I wouldn't be surprised if an multi-node IMU-based system could have low latency. After all, the Rift's headtracker is quite good.

I'm with you that this is probably the best solution for large environments particularly when limb tracking is required. However, you still might want another system for at least rough absolute positional tracking, as it's still technically possible to introduce drift in this system.

Yes, I suspect that very low latency is possible with this sort of setup.

Agreed on the need for external absolute tracking for overall position. High end optical systems are expensive and complicated because they use a unified system for both local and global tracking. But since the two types of position can have very different specifications, splitting the job between two systems can have some significant cost benefits. Often the local system needs low latency and high accuracy but can be drift tolerant. The global system cannot drift but can have high latency and low accuracy. This is the basic compromise I sought with the Red Rovr project. Inertial local tracking combined with GPS global tracking, but there are other systems that you could use. Particularly if you lower the requirements for global tracking you open the door to low resolution optical (giant ball on your head), large scale magnetic variance (good in steel buildings), RF power triangulation, etc...

[blazespinnaker]

My experience with what I've done so far is that latency isn't a big concern. IMUs can report very fast and you can generally predict given changes in acceleration what the user is about to start / finish in terms of actions.

I think there is a great future in IMU motion detection:

http://www.youtube.com/watch?v=oWu9TFJjHaM

I suspect the iWatch (or whatever) will be all about this. Imagine starting your car or opening the lift gate with your watch/smartphone/sensor detection? The potential is really amazing. May the force be with you!

Edit: I agree on the positional. That's why I actually think it's better as a gesture recognition input device more than a motion-capture system, at least for now.

[brantlew]

My experience with what I've done so far is that latency isn't a big concern. IMUs can report very fast and you can generally predict given changes in acceleration what the user is about to start / finish in terms of actions.

Not sure I can agree with this statement - at least not in the general case as demonstrated in this video. A basic binary pedometer can be achieved with a great deal of accuracy and responsiveness - yes. But as you allow for full freedom of motion, it becomes exceedingly difficult to differentiate between all the different modes. Even something as seemingly simple as a crouch looks maddeningly similar to the beginning of a walk, and you have to really push your filter windows way out and take on huge amounts of latency to discriminate the two actions.

[blazespinnaker]

Not sure I can agree with this statement - at least not in the general case as demonstrated in this video. A basic binary pedometer can be achieved with a great deal of accuracy and responsiveness - yes. But as you allow for full freedom of motion, it becomes exceedingly difficult to differentiate between all the different modes. Even something as seemingly simple as a crouch looks maddeningly similar to the beginning of a walk, and you have to really push your filter windows way out and take on huge amounts of latency to discriminate the two actions.

Well, I was mostly talking about my experience, but perhaps that's because my application is measuring a pretty narrow and well defined range of motions (which I think is an OK tradeoff if you only have one IMU)

But still, if we're talking about the more general case, it seems to me if you have sensors in your feet versus sensors on your torso, it should be pretty easy to differentiate between walking and crouching (taking your example). Not in isolation, of course, but combining the information you get between the different sensors.

If your goal is a full on motion capture system, then I suppose the tradeoff might then be between # of sensors and latency (assuming it doesn't overwhelm computational / bandwidth).

[brantlew]

If your goal is a full on motion capture system, then I suppose the tradeoff might then be between # of sensors and latency (assuming it doesn't overwhelm computational / bandwidth).

Yes I agree. IMU based motion is very promising but only with a sufficiently large sensor network. I even think it can suffice for motion capture provided you are not attempting true environmental interaction with props and other people. And even below the sensor network threshold you can still do some interesting things, but only with some major constraints and compromises.

[zalo]

Hybrid magnetic/IMU systems could be interesting too.

Instead of using the feet as the root, you could use the HMD w/ an absolute tracker as the root and work your way downward with IMUs on each limb.

You'll need fewer IMUs to get the body points of interest.

Though 5 wireless magnetic trackers might be less expensive than ~15 wireless IMUs...

[blazespinnaker]

So .. .any experience with FreeIMU? What are folks using to build their own sensor networks?

http://www.varesano.net/projects/hardware/FreeIMU
https://viacopter.eu/multirotor-shop/se ... pter-store

Interesting "smart skeleton" project:

http://www.jpattillo.net/?page_id=72

[brantlew]

As far as I know, you've got to get your hands pretty dirty to build a sensor network. YEI has a full kit but it's a bit expensive. For ~$1000 you can probably get some off-shelf sensor PCB's (ie Sparkfun) and built a decent little network or if you want to get your hands really dirty with raw components you can probably go < $400. But wiring and firmware skills are going to be required for this type of effort.

[blazespinnaker]

Interesting. I wonder what the economics look like as you scale. This seems like the right approach for mocap input on VR games. Kinect suffers from facing away and your body occluding the camera.

Though I suppose you could have a multiple camera setup. That could be more ideal than having to clip sensors on to your clothes. It could also deal with multiple players and less positional drift.

http://www.youtube.com/watch?v=LKi1nEDeZJc

[brantlew]

Kinect is good for setup - although as you start adding cameras those advantages diminish a bit. Plus you are still dealing with all the resource load, latency, and occlusion issues that come with vision. The advantages of sensor networks start to look very appealing in comparison - except for the giant hurdle to cross of a wearable system. I always thought a jump-suit type setup would be an ok compromise. Just pull it on and zip it up. Not too much hassle although you are going to have to be pretty dedicated enthusiast to literally want to "suit-up" in VR.

Edit: Oh yeah, and the other nice thing about sensor networks is their practically unlimited range. Very handy for someone like me who likes to run around in large scale environments.

[blazespinnaker]

http://sixense.com/wireless

I wonder how much sixense is throwing their hat in the ring with the new hydra for full body mocap. That clip could be useful for single IMU mocap .. maybe even better if they do fusion with the hand controllers. Plus it'll probably have positional.

Those two empty ports could be spots for shoe clip ons as well.

Pretty interesting development.

[zalo]

Hybrid magnetic/IMU systems could be interesting too.

Instead of using the feet as the root, you could use the HMD w/ an absolute tracker as the root and work your way downward with IMUs on each limb.

You'll need fewer IMUs to get the body points of interest.

Ahem.

A shirt tracker means you only need two IMUs to get the movement of a whole arm. Four for two arms, plus little IMU controller gloves for the hands, the IMU in the rift, and you have the whole upper torso! (This is pretending you're not already using the controllers with the hydra-ense)