Meant to be Seen

I thought this might deserve its own thread...

[youtube-hd]http://www.youtube.com/watch?v=LDgINNDDmVs[/youtube-hd]

The video shows a ps3 eye being used to track and identify LEDs which are being controlled by an arduino.

The arduino varies the intensity of each led according to frequencies which are unigue to each led.

In the video you can see the graph of the fourier transform which shows the frequency components of the selected LED.

By forming the graph of each LED and comparing it to a known frequency spectrum the app works out which led is which and tracks the selected led from frame to frame.

This is great for my optical tracking project, but I thought the idea might be useful to others...

The source will be in my tracker soonish...

Very nice.

I wonder if this is more or less susceptible to noise than say - differentiating visible spectrum LED's ? I guess it's less susceptible to ambient lighting fluctuations.

If you do it with nIR LEDs, along with the appropriate camera filter, then in indoor environments you can almost rule out noise. Close your curtains and avoid incandescent bulbs in the camera's FoV, and the only sources should be your markers.

Using the visible spectrum LED of the PS Move with an unmodified PS Eye gives surprisingly good results, here is a capture in my room with several lights on using the PS Move API :

I should mention these are IR LEDs but they could be emitting anything...

The purpose of this setup is to uniquely identify each LED, so that I can tell the back of the head from the front, the left from the right and so on. This will also be useful for separating out multiple users in a space and say a gun from a helmet.

The potential improvement in filtering might also be useful...

Nice work. You could probably combine this with different color / IR range filters, you would probably need to for VR lasertag with lots of players, especially if you wanted to track their global location around the arena.

I agree to scale this up to hundreds of leds, for something like vr laser tag, some tweaking would definitely be required. I was considering switching to the visible spectrum and using something like this:
http://www.makershed.com/LoL_shield_for ... /mkjr3.htm

This should allow a lot more information to be transfered, by varying each led independently. I havent really looked into encoding schemes as a simple cosine similarity measurement between the frequency spectrum observed and a reference frequency spectrum is probably enough to take me up to 10 leds. A simple scheme I am considering is using a prefix code transmitted on a specific frequency, but this is a deep topic and I know very little. I think I will design it if I need it.

With the led identification I have so far my next job will be to use the leds to do orientation tracking of the head. Depending on how that works, I might start to integrate a kalman filter into the led tracking, as I suspect I will need something like this when I come to do sensor fusion with the rift 6/9 dof tracker.

Prob too much info, but hey ho

MemeBox wrote:I agree to scale this up to hundreds of leds, for something like vr laser tag, some tweaking would definitely be required. I was considering switching to the visible spectrum and using something like this:
http://www.makershed.com/LoL_shield_for ... /mkjr3.htm

This should allow a lot more information to be transfered, by varying each led independently. I havent really looked into encoding schemes as a simple cosine similarity measurement between the frequency spectrum observed and a reference frequency spectrum is probably enough to take me up to 10 leds. A simple scheme I am considering is using a prefix code transmitted on a specific frequency, but this is a deep topic and I know very little. I think I will design it if I need it.

With the led identification I have so far my next job will be to use the leds to do orientation tracking of the head. Depending on how that works, I might start to integrate a kalman filter into the led tracking, as I suspect I will need something like this when I come to do sensor fusion with the rift 6/9 dof tracker.

Prob too much info, but hey ho

I would use CDMA with the clock of the LEDs synced to the vsync line on the camera. It helps that you only need to identify LEDs when you initially see them or need to disambiguate after overlap. It then matters less that it takes more frames to uniquely identify them.

I'm not sure how effective this can be unless your update rate is really low. If the pulses driving the LEDs are much faster than the frame rate of the camera you probably just overall reduce the brightness of the the LED in the frame, or you introduce strange nonlinearities during pixel scaning. Lets say the pixel clock of the imagine sensor and the pulses to the LEDs are the same, and pixel 1 and 2 of the imaging sensor should be able to detect the LED, you might get a scenario where only one of the pixels is able to detect it while the other one doesn't since the LED is in the off state when it's being sampled. I image it gets worse when you add motion to the mix. Maybe I'm missing something