Nice find. Would any of those fit?FingerFlinger wrote:In case it wasn't already common knowledge, Peau Productions has a selection of wide FOV lenses for the PS3 Eye.
I wonder if there are focusing problems with it? Would the stereo images inhibit auto-focusing?For VO auto-focus might be bad. In theory, if you know the focal length you should be able to compensate for the changes. I have always worked with fixed focal length lens. My recommendation would be to turn off auto-focus until the very end when everything else is working.brantlew wrote:Nice find. Would any of those fit?FingerFlinger wrote:In case it wasn't already common knowledge, Peau Productions has a selection of wide FOV lenses for the PS3 Eye.
So now we're basically talking about a perfectly synchronized stereo camera with a 55 degree FOV, 320x480 resolution, 60 FPS, with about a 5 1/2 inch baseline. Sounds really promising to me.
For now I am sticking with outward looking tracking or a combination with an IMU. This is also closest to our own senses and any other types of tracking with external peripherals would only distract us from the optimal solution of walking around freely.MSat wrote:Outwards looking optical tracking seems extremely tricky, and agree with whoever suggested small areas would probably be better served using an inwards looking system. There's a lot less calibrations and computations involved.
... ideally in the sub-millimeter, sub-degree, 60+ Hz range. A pretty tall order
Don't forget to add 2 more parameters into that equation, latency and wireless.mahler wrote:For now I am sticking with outward looking tracking or a combination with an IMU. This is also closest to our own senses and any other types of tracking with external peripherals would only distract us from the optimal solution of walking around freely.MSat wrote:Outwards looking optical tracking seems extremely tricky, and agree with whoever suggested small areas would probably be better served using an inwards looking system. There's a lot less calibrations and computations involved.
The real-time Stereoscopic Visual Odometry example in the BoofCV library and the Loreo 3D Lens in a Cap to use cheap hardware for synchronization gives me confidence that we're getting close to solving the issue of ideal positional/orientation tracking for VR.
But as brantlew said:... ideally in the sub-millimeter, sub-degree, 60+ Hz range. A pretty tall order
You can feed the orientation estimate from the IMU into the feature tracker and use that to simply/improve feature tracking. It works really well too. If you want to get really fancy you could use the vision algorithm to remove gyro drift when stationary. I'm not aware of any open source projects that support this functionality (boofcv doesn't yet). Maybe LIBVISO does?brantlew wrote:Yeah, ideally in the sub-millimeter, sub-degree, 60+ Hz range. A pretty tall order - but that's if you rely completely on the camera. There has been some speculation that a sensor fusion with an IMU device could lower the camera requirements.
Since IMU's provide very good angular data, is there a way to "remove" the angular calculations from the VO algorithm and just track position? Would this lessen the computational requirements in any way - or would pretty much the same amount of work need to be done just to compute position with any sort of accuracy? Oh, and another thing. For this specific VR problem, scene reconstruction or accurate depth mapping are unnecessary. Would removing those requirements allow the VO algorithm to be reduced?
Great! That coincides with what a computer vision engineer friend of mine was telling me - IMU's and CV complement each other well because they have opposite strengths and weaknesses. If you know of some good research and/or papers on the subject I would love the links.lessthanoptimal wrote:You can feed the orientation estimate from the IMU into the feature tracker and use that to simply/improve feature tracking. It works really well too. If you want to get really fancy you could use the vision algorithm to remove gyro drift when stationary. I'm not aware of any open source projects that support this functionality (boofcv doesn't yet). Maybe LIBVISO does?
I recently saw an IMU with gyro and acceleration that was about 1"x1"x1". What's really notable about it is that the acceleration was accurate enough that you could integrate it and estimate your 6-DOF pose for a little bit. The price I believe was several hundred. Probably more than you guys want to pay, but really good compared to what something like that would cost a few years ago.
One warning about VO or structure from motion (SFM) in general, you can't do a half ass job at it. There are many things that you have to get right for it to work, but if you mess up one of them it will kinda work but fail easily. I'll post a reading list for you guys.brantlew wrote: I've currently got a really, really shitty prototype version of a CV position tracker.
You are 100% correct there. A major criticism of VO is that its performance is highly dependent on lighting conditions and the amount of texture in the scene. Classic failure conditions are when the sun is low in the sky or textureless hallways indoors. Any systems which uses VO will need to be able to handle interruptions in motion data. In the future hallways might not be as much of an issue, but it will require better image features than are being used now.brantlew wrote: One thing that is striking to me is how incredibly sensitive it is to the environment. Lighting and scene content seem to have a massive effect on the speed and performance of the algorithm.
Haha yep. It's also amazing how there is never sun glare blinding the camera. I was debating if I should use more challenging datasets in my propaganda videos. Ended up deciding to leave that for a future video. There was also the problem of finding a really bland office to film in. I was unable to get permission since I wanted to post the video on the web.brantlew wrote: There's a reason that all those youtube demo videos are filmed in a messy office.
Chriky wrote:I agree that I don't think arbitrary algorithms are there yet, or, we don't have the hardware (cameras and processors) to make the current algorithms work well enough. I like the idea someone had of projecting bright dots around the room with a laser projector. It would let you control the environment to a certain extent without being inconvenient.
I don't think its unreasonable, at this stage, to require people to dedicate a bit of time and space to VR, it just needs to be cheap and easy.
Chriky wrote:I agree that I don't think arbitrary algorithms are there yet, or, we don't have the hardware (cameras and processors) to make the current algorithms work well enough. I like the idea someone had of projecting bright dots around the room with a laser projector. It would let you control the environment to a certain extent without being inconvenient.
I don't think its unreasonable, at this stage, to require people to dedicate a bit of time and space to VR, it just needs to be cheap and easy.
Good advice - and my implementation isn't even half-ass. More like 1/10 of an ass.lessthanoptimal wrote:One warning about VO or structure from motion (SFM) in general, you can't do a half ass job at it. There are many things that you have to get right for it to work, but if you mess up one of them it will kinda work but fail easily.
I did a big research binge on OF sensors recently, the idea being to glue 6 dirt-cheap sensors onto the HMD looking in opposing directions on all axes in order to get the overall optical flow for the entire head using a fairly weak microcontroller to do the basic algorithms required.MSat wrote:The optical sensors in mice, while being extremely fast, have VERY low resolutions (30x30 is considered "high-res") and mediocre monochromatic color depth. I don't think they'll do much good mounted to your head unfortunately.
I should have mentioned that I was talking about sensors with an appropriate new lens (fixed focus or EDOF). There are a few papers on these sort of 'hacked' sensors, e.g this one on fixed-range 1d odometry, or this one showing just how small a range of distances are viable with a fixed lens. Unfortunately, I couldn't find any papers on use of COTS OF sensors with telecentric lenses, probably because telecentric lenses seem to be custom made in most cases, or at the very expensive enough that pairing them with cheap sensors is a false economy. 'Small cheap plastic telecentric lenses' don't seem to be an actual thing.brantlew wrote:@EdZ: I had that intuition as well. Mice are designed for a very constrained problem. You can't even lift them up 2 millimeters and maintain tracking.
While the optics definitely play a role in this, the functionality to cease response is actually part of the sensor's firmware, and can even be changed in basically real time on some of them.brantlew wrote:@EdZ: I had that intuition as well. Mice are designed for a very constrained problem. You can't even lift them up 2 millimeters and maintain tracking.
Good idea, or maybe just use a Kinect as your "dumb" IR shape projector, it wouldn't necessarily need to interface with your IR camera tracking solution,brantlew wrote:It would be great if a home planetarium projector could be used. Point sources would be easy to deal with if they were bright enough to be imaged reliably. You would need a really strong projector and it would only work in the dark and your camera would accumulate a lot of noise in that environment. Maybe a solution would be to swap out the bulb with a strong IR light source and then put an IR filter on the camera.
There are rumours floating around about Kinect 2.0. I would be surprised if some of the 640x480 or 30 fps figures do not go up.MSat wrote:Hmm.. So the Kinect uses a imaging sensor @ 640x480 and 30fps. Would there be any benefit in using the laser projection with a faster (and maybe higher res) camera, or is the algorithms is uses too slow to be of any benefit?
) But dense fields increase the search space. For stationary usage I think maybe a less dense field would be better. Not only does it decrease the search space, but it would seem to allow faster head movement with slower cameras since large point movements could be more easily correlated from frame to frame. In fact, (except for latency) slower cameras with longer exposure times might even be helpful with sparse fields because you could actually capture the optical flow direction of the points in the motion blur of the image. You just need to make sure your points are super bright and the resolution of the camera is good enough to detect them dependably in every frame. I've got a buddy that does laser shows in clubs looking into programmable IR laser projectors. Great, that's good to know, I might have a go a this when my second PSeye turns up. What's your take on using a projected pattern to assist tracking?lessthanoptimal wrote:It links to a few other message boards where people had gotten frame synchonization working by hacking the hardware on those bad boys. So that issue might be solved.
I assumed that "swirl" was intentional, to disrupt the repetition of the base pattern, but I think you're right, it's probably just a sampling artifact! There's a bit more on the patterns here:brantlew wrote:I can't tell how much of the "pattern" is due to positional offset and how much of that is just due to intensity gradients or moire.
I've seen similar "long exposure" tricks used in astronomy imaging, we could probably learn a thing or two from those guys,brantlew wrote:In fact, (except for latency) slower cameras with longer exposure times might even be helpful with sparse fields because you could actually capture the optical flow direction of the points in the motion blur of the image.
Hmm.. It looks like the camera boards are just mounted to some scrap PCBs with some standoffs. If you don't need ultimate precision and adjustability, you can probably just mount them to a single PCB (get one of those prototype boards from radio shack), or some aluminum L-angle. I was thinking that maybe with two high-bandwidth USB2.0 cams (like the PSEye), you might want to connect them to a USB3.0 hub to try to avoid potential bottlenecks.brantlew wrote:
Here is a more elegant solution. Looks great and the sensors should be perfectly aligned. But where do you get those square mounting PCB's? Those are not part of the camera. The camera PCB is that little wing shaped board. What all parts would be necessary to assemble this?
http://www.flickr.com/photos/60552763@N00/5089295727/
