IR tracker

[MemeBox]

I've made an optical IR tracker:

[youtube-hd]http://www.youtube.com/watch?v=3IGdW2Vx4VI[/youtube-hd]

It would be great if it wasn't such a lonely exercise, anyone else fancy helping?

The plan is to use this to enable 6DOF tracking for the Oculus...

More details over at: http://www.mtbs3d.com/phpbb/viewtopic.php?f=138&t=15474
Link to source (early prototype), is buried in that thread.

[cybereality]

Interesting, but I'm not exactly sure what I'm looking at.

Can you take a picture of the actual hardware maybe?

[android78]

I love the mapping of the room, that seems to have worked better then I expected. So you are just holding (or on your head) your ir led lights and moving them round your room to move the virtual camera in the video? Seems pretty smooth. How do you find the lag? Also, how do you determine orientation (as opposed to position)?

[FingerFlinger]

Cool man. I've been keeping an eye on your other thread, because I wanted to start a similar project but never got to it. Curious what kind of occlusion situation you've got. From your video, it looks like the user would have quite good freedom of motion, far better than TrackIR, at least. Also, in your other thread, you mentioned something like a 3 meter range. Is that a hard-and-fast number, or is it just that the accuracy fall off at around that distance?

Also, what is your current hardware setup like? Are the 2 PS3 Eyes synched to each other in hardware? How are they positioned?

[MemeBox]

I have 4 PS3 eyes placed in the four corners of my lounge.
Each camera faces towards the centre.

In the video I am moving around my lounge with the light source, showing different angles of the table, sofa and 3D model I have imported.

The black points are a simple mapping of my lounge, formed by using a laser pointer. I moved the laser pointer around my lounge, which allowed me to locate the 3D position of the dot using the camera setup. As the dot plays over the surface of things, it maps out their location.

The light source is tracked in each camera and then some funky maths converts those image locations to a 3D location for the light source.

To get a 3D location a track from at least two cameras are required. If the light source is occluded in some of the cameras the track is maintained by the remaining cameras.

There seems to be little lag, I can just barely perceive some lag when I throw the light source onto the sofa and try to match the sound to the impact visible on the screen. I have little experience, but the cameras are running at 60fps and you might expect therefore ~16 + x ms delay where x is the time it takes to transfer from the ps3 eye to the computer. Once it's there the processing is done in time for the next frame, so this should be sub 16ms. There is no hardware synchronisation, but it appears not to be an issue. If the light source is moved fast, like pretending to serve in tennis or play a golf swing, you can see some jitter in the tracked points, I think this might be due to frames not being captured at exactly the right time, but it's not a problem for my intended application...

I am tracking at 60fps, the video looks jerkier, but this is due to the video encoding.

You can see a slight 'jerk' of the camera, in the video, this is because I have blocked the view of one or more of the webcams with my body. The amount of jerk you get is dependent on the quality of the calibration, this is not the best calibration I have made, I did not give it much attention, hence the jerk. But this should be less of a problem when the light source is on the head and not being waved around in the hand.

The area over which this can operate is limited by the distance over which the cameras can see the light source. So with my current light source I would be surprised if it wouldn't work over the area of a football pitch. But it all depends on how much Infrared light is around, in bright day light the current setup is never going to work. Also the precision of the 3D location will decrease as the area increases. Adding more cameras will increase the precision. Exactly how big a space is workable I don't know...

Calibration of the cameras is automatic and requires only that you move around the space with the light source. About 30 points are required for a calibration and this takes about 5 seconds to perform.

I am not dealing with orientation, the camera is just set to look toward the model. I have some plans for this, but I'm waiting on some IR leds and some modelling clay. It may be that I have to rely on the orientation reading from the Oculus.

I will post some pictures tonight, when the day job is over.

[MemeBox]

This new video shows the setup:

[youtube-hd]http://www.youtube.com/watch?v=sEx1nNkEW_o[/youtube-hd]

[zalo]

That looks cool! What do you think the accuracy would be like with just two cameras? Good enough for positional head tracking?

Also, I encourage you to show us more, especially with the calibration stage and the "room mapping"
And please don't make the camera always look at the model, it makes it hard to see just how accurate and responsive it is.

You have a really good system here, don't give up! I expect interest in positional head tracking will spike after the Rift is everywhere. Because of the on-board IMU, orientation is taken care of, so position will become critical.

Also, how well do you think single point tracking will work over long distances and big spaces? Say VR laser tag with Magnetometer adjusted orientation and Optical position...

[cybereality]

Nice!

[MSat]

Very slick! Particularly the way you're mapping objects in the room.

[FingerFlinger]

Very nice! Are you limited to 60 fps because of processing time, drivers or USB throughput? Although from your video, the lag certainly seems acceptable.

[brantlew]

Very nice. That's a really bright light and it looks to be in the visible spectrum. What is it?

[MemeBox]

This shows the calibration routine and later on shows the tracking without the camera automatically centering on the model.

[youtube-hd]http://www.youtube.com/watch?v=FAD5OZa59fE[/youtube-hd]

The light is a torch, but it might as well be an IR source. It was an led a while ago, but I used up all my Leds gluing them to a helmet, to try to do orientation tracking...

[brantlew]

What do you mean by "torch"?

[Fredz]

Flashlight in American English, torch is British English.

[MemeBox]

A bit more info:

The fps is currently limited by the PS3 Eye max framerate at 640*480 resolution.
I believe some of the jitter in the video is introduced by the flare around the large bright light source, it seems to work better once the battery starts dying a bit. A finer LED source should work a little better, not radically, but better.

In answer to laser tag over large spaces:
I can't work out at the moment how the resolution increases as cameras are added.

But with this setup I could have 4 cameras in my lounge, 4 in my hall way and 4 in my bedroom and calibrate them all into the same space, as long as there are points of overlap, where some cameras from each group can see the same calibration points (points grabbed from moving the light around).

[Okta]

A bit more info:

The fps is currently limited by the PS3 Eye max framerate at 640*480 resolution.
I believe some of the jitter in the video is introduced by the flare around the large bright light source, it seems to work better once the battery starts dying a bit. A finer LED source should work a little better, not radically, but better.

In answer to laser tag over large spaces:
I can't work out at the moment how the resolution increases as cameras are added.

But with this setup I could have 4 cameras in my lounge, 4 in my hall way and 4 in my bedroom and calibrate them all into the same space, as long as there are points of overlap, where some cameras from each group can see the same calibration points (points grabbed from moving the light around).

Then it seems like you really have the ingredients for VR lasertag etc. Your system provides the players positions and the rest is taken care of by their personal trackers and controls. Great work man.

[Nick3DvB]

Flashlight in American English, torch is British English.

...or maybe just English english!



Great work MemeBox, I will have to try this out soon, any more progress to report?

I notice the github hasn't had an update for a while: https://github.com/MarcusRobbins/Free3DTrack


BTW - if you need some more PSeye cams most GAME stores are doing them 2nd hand for £2 each!

Are you removing the IR filters? If so make sure you get the right model version:

http://www.peauproductions.com/blog/200 ... s-of-each/

[MemeBox]

I will update the git version tomorrow, the newer version is quite a bit better. I am removing the ir filter and game has been the source for all of the ps3 eyes! Although mine were £4 each. I learned the hard way about different versions of the ps3 eye. I am waiting for my oculus to arrive before I do more work on the project, it does what it should at the moment, more work may be a waste. If you want to get the software working, wait for the newer version, and ask me any questions you need!!!!

[Nick3DvB]

Cool, looking forward to the new version, will have to pick up a few more PS3 eyes first though. I wonder if adding an IR (only) pass filter would help at all, the FreeTrack guys just slip a piece of old floppy disk or exposed film behind the lense, might be worth a try?

[MemeBox]

I've just tried to push the new version to Git, but it isn't happy, I think it's my workplace internet connection. I will push it when I get home.

I am using exposed film to block visible light while leaving IR light.

When you remove the IR filter from the PS3 eye, it ruins the focus. The IR filter also seems to be a lens. But there is a lens behind the IR filter which is movable. If you rebend the plastic that you bent to remove the IR filter, back into place, you can re-position the free lens so that you get a fairly sharp focus...

Good luck with it, once you feel like having a go and have a copy of the software, drop me a message and I can help you get setup...

[Nick3DvB]

Thanks! My local GAME is all out of cams, will pick one up on ebay asap.

[MemeBox]

I have uploaded the newest version of the code to Git.
Drop me a line, anyone, who wants to get this setup...