A question on positional tracking

[adam.wilko]

Hi all. First post here.

I had a question about positional tracking, since all the big players are focussing on camera based tracking, which leaves mobile a bit out in the cold.

I understand the problem with IMUs not being able discern between tilt and acceleration in a lot of scenarios, but what I don't get is why can't this be solved by simply throwing more IMUs at the problem?

In the case of the rift and the gear vr, you have straps around the sides and over the top of the head. If you had 5 IMUs mounted at the front, back, top, and both sides of the head, all on the same rotational alignment and as widely spaced as is practical, they're all going to give very similar readings for translational acceleration, but wildly different readings for rotational acceleration. Couldn't these be very easily cross referenced to give extremely accurate full 6DOF tracking?

I guess my question is twofold, if it would work why isn't everyone doing it? and if it wouldn't work, can anyone explain exactly why it wouldn't?

[cadcoke5]

The reality is that they are NOT that accurate. They tend to drift a large over time. The only acceleration that does not drift, is gravity. Though even that shifts if the person moves their head in any direction other than purely left-right.

Absolute position methods are necessary. There are a variety of methods for absolute position sensing, and each have their weaknesses. If it is optical, the target may be obscured. During those moments the inertia sensors can fill in the position data.

-Joe

[adam.wilko]

Thanks for the reply. I posted this same question elsewhere and someone linked this video, which shows why a single IMU doesn't work, but it also hints at why Oculus and others wouldn't have pursued a multiple IMU solution: If one thing works well (gyro + magneto) and the other doesn't, the solution is not usually to throw more of the non-working thing at it, right?


https://www.youtube.com/watch?v=_q_8d0E3tDk


I'd argue that positional tracking doesn't need to be in absolute coordinates and 100% accurate to effectively reduce motion sickness, and where you have glitches caused by sensor occlusion etc. the cure is sometimes worse than the problem, which was my experience when comparing DK2 camera based tracking glitching out when hands are raised vs. strapping a moderately accurate hydra to a DK1.

Acceleration accumulating into velocity accumulating into position means that a tiny error at the start becomes huge over time. You get this in physics games whenever the drag is zero, and the general solution there is to simply add a bit of drag, so it it occurs to me that positional drift derived from pure acceleration input could be dissipated reasonably effectively before it takes hold with just a little per-input velocity damping, obviously not effectively enough though, or we'd already be doing it.

The down side of adding drag is that it may zero out extremely slow and steady movement, but I think there would be a threshold where fine motion response vs error quashing would be very tolerable, not perfect, but good enough to quell motion sickness and much better than nothing.

That's where the multiple accelerometers come in. They could immediately detect and remove most of the tiny errors at the source before it is integrated into the velocity at all, allowing you to use much less drag to zero out the drift.

If you take for granted that the sensors cannot change their position relative to each other - they are in fixed positions around the head and must move as a whole - you could use something similar to Verlet integration with a little error tolerance to hold their positions in place, maintaining the same distance from each other and the same distance from an imaginary centre point which would more or less average out the error on each input as it comes in.

I say "more or less" because you'd find the best fit for the majority of the sensors and pull the minority into line with that. Each sensor would use the most reliable 3 of the other 4 sensors to recalibrate itself on a per frame basis, and when there is consensus between 4 or 5 sensors that movement occurred that movement be passed on to the camera. Where there is less consensus, movement would err on the side of deceleration, thus reducing drift under error conditions.

Of course, all of this would be cross referenced with the gyros - you know the rotation angle, so the difference in acceleration at points around the head will give you a centre for the rotation, subtract the acceleration caused by that rotation and you are left with a translation per sensor that should in theory all be the same for all sensors. Where the vectors are consistently different you can work backwards to determine differences in mounting angles of the sensors, and you could even take it a step further and use something like Baye's over time to calculate the exact coordinates of the sensors relative to the centre and each other after just a few seconds of input, since the strap-mounted sensors will be positioned differently between uses and between users. This refinement of sensor placement would feed back into the error absorption loop, giving greater accuracy once it figures out exactly where the sensors are.

The more I think about this the more viable it seems, I haven't really done electronics since I was a kid though.

I'm thinking the first step would be to set up a little test in software that randomly jitters acceleration of 5 points and see where to put the drag and threshholds etc. for varying amounts of noise. I guess a lot comes down to the specifics of the inaccuracy coming from the sensors though, only way to test it is build one I guess.

[WiredEarp]

Acceleration accumulating into velocity accumulating into position means that a tiny error at the start becomes huge over time. You get this in physics games whenever the drag is zero, and the general solution there is to simply add a bit of drag, so it it occurs to me that positional drift derived from pure acceleration input could be dissipated reasonably effectively before it takes hold with just a little per-input velocity damping, obviously not effectively enough though, or we'd already be doing it.

I dont know enough about the issue, but doubt that would be a solution. I suspect you'll need to zero it to a known position/velocity to be able to achieve accuracy, damping will not have this known position/velocity. Most systems that use accelerometers for positioning successfully rely on resetting it when its stationary (ie, a heel sensor on a foot based system). I've heard a few people say the Rift DK2 system uses accelerometers for position, and only uses the camera for position resetting, but I'm not sure if thats true going on how my tracking glitches out immediately when I cover the camera.

Of course, all of this would be cross referenced with the gyros - you know the rotation angle, so the difference in acceleration at points around the head will give you a centre for the rotation, subtract the acceleration caused by that rotation and you are left with a translation per sensor that should in theory all be the same for all sensors.

To simplify this logically, isn't this the same as if the head has zero rotation? I dont see that the rotational information will help at all with figuring out the translations, so i'd ignore that for your testing personally. The problem is, you are going to get a translation for each sensor, that will extremely quickly display accumulate drift. These errors will accumulate within 20-30 samples, so < a second of data.

Basically, I dont see how adding more sensors is going to help the accuracy of translational sensing at all. The angular sensing will have zero benefits (at least as far as I can think about it now, i'm at work, so perhaps haven't had a chance to give your post all the attention it deserves), and you haven't really mentioned any way to reset the sensing drift beyond damping it (which I dont _think_ will work unless you are sure the head is stationary). Even with multiple IMU's, I think they ALL will be inaccurate enough that you will not really be able to average them or use them in any way to get greater accuracy. However, experimentation always beats armchair theorizing, so I'd be interested to hear about any results you might come up with.

[adam.wilko]

So I did a bit of tinkering, just emulating things in software with Unity, as I don't have the requisite knowledge to put a board together.

First, I have a position with a single small acceleration value being added to it per frame, it very quickly accelerates out of control, similar to the video above, but I made it drift even faster.

Then, for the hell of it, I tried iterating the sensor error 5 times and just using the average, reasoning that additional samples would make a parabolic distribution and that would smooth things out, and sure it helped a little but not enough. I had to add at least 20 iterations before seeing decent stability, and even then, it depends entirely on the errors being properly random, which is probably not the case with real sensors.

Next, going back to assuming that the default state of the head is stationary, instead of taking the average of all simulated sensor errors, I took only the one that has the least acceleration / most deceleration - in other words the one with the lowest dot product vs the current velocity. With just 3 sensors it is pretty good, but with 5 it immediately becomes very solid and stable. With error amounts amplified to the extent that the single sensor version exceeds the limits of Unity's coordinate space in just a few seconds, the 5 sensor version has zero drift.

The downsides: After I add some velocity manually with the arrow keys, it takes quite a while (maybe 10 seconds) but the errors do eventually accumulate, only now they accumulate to stop the motion entirely instead of driving it faster. This also means that under real acceleration you would only move as fast as the slowest sensor.

This is before you even start having the sensors compare notes to detect and counter the errors, or using gyro and rotation to cross check positions with a bit of trig, let alone applying some simple machine learning to the whole thing.

So, yeah. Without physically building something I cannot be sure. Maybe the sensors will all error in the same direction at the same time because of some common internal mechanism? Maybe physically mounting the sensors at different angles would be enough to counter that?

In any case, just taking a larger sample size and erring on the side of deceleration is extremely effective, at least in the simulated case where the input error is random. It is so effective that it is pointless for me to try more advanced solutions to further correct drift, because there isn't any drift with 5 sensors.

I suspect that sensor error will have physical causes behind them and not be random, and also guess the zero point might get a little offset, so it might report the same or similar vector many times in a row. If that is the case, and the sensors are all differently angled it should be very easy to spot the miscalibration and continually re-zero each sensor at 1000Hz by storing the current known errors and using them as offsets on the next set of inputs.