Visual Odometry/Position Tracking Worklog [Computer Vision]

This is for discussion and development of non-commercial open source VR/AR projects (e.g. Kickstarter applicable, etc). Contact MTBS admins at customerservice@mtbs3d.com if you are unsure if your efforts qualify.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Visual Odometry/Position Tracking Worklog [Computer Vision]

Post by FingerFlinger »

For the past 7 months or so, I've been doing a lot of computer vision research, and I think that I have been zeroing in a pretty good solution for position tracking. I thought I was pretty close to showing off something really cool, but unfortunately, most of my work exists in the form of really hacky, one-off research projects. I spent the last week trying to reuse some of that code, but it's just not going together easily.

So, since I am finally beyond the point of aimless experiments, I'm going to do a proper design, and make a releasable project. I probably won't release all of the code, but I'll post what I can along with the binaries.

I want to keep a sort of dev log here just to encourage myself. Motivation is not actually the issue, (I've been working like a dog) but it's sort of depressing to not be sharing my work with anyone.

To start off, I just uploaded a video showing one of the core concepts of my algorithm, which is to use raw optical flow results in combination with a conventional reprojection algorithm. From my experiments it GREATLY increases the precision of the tracking.

http://www.youtube.com/watch?v=LmN70tDTHo0

EDIT: Also worthy of note is that the demo in the above video is using no smoothing filter on the results; it's literally a single frame's worth of raw OFlow data averaged together and using outlier rejection .
Krenzo
Binocular Vision CONFIRMED!
Posts: 265
Joined: Tue Sep 07, 2010 10:46 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Krenzo »

Have you considered the case of the user looking at a blank wall? It looks like if you were to stand up and not see your monitors anymore, the tracking would fail.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

That's an issue with any visual solution. If there are no interest points in the camera's view, then there is nothing you can track.

More cameras or a larger FOV can mitigate the problem, and is something I'll explore eventually, but it's an implementation issue that is not related to the algorithm.

I'll also note that any motion on the monitors will affect optical flow results, in general. You will reject most of it because the motion will register as an outlier, but if the monitors instead showed a static image, it would contribute valuable information to the algorithm. So, at best, any motion on the screen will take away from the total information that would have been available to the tracker.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by geekmaster »

FingerFlinger wrote:That's an issue with any visual solution. If there are no interest points in the camera's view, then there is nothing you can track.
Could that be why photos of Valve employees often show visual odometry fiduciary markers sprinkled over the blank spots on their office walls?

Image
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

I would guess. The other benefit of that is that you are salting the room with unique markers, so you don't need to really map the room.

Without mapping, or external tracking, any dead-reckoning system will drift. But if you can identify a fiduciary marker with a known location, you can reacquire the camera's absolute location.

This would eliminate a lot of overhead from mapping, but you have to get users to actually do it...

EDIT: Related to the user compliance issue is that people game at all times of day, and may or may not be playing in a well-lit room. This, obviously, is a huge problem for a passive tracking system, because the camera will be blind.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by cybereality »

Looking good man!
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by brantlew »

Very nice Finger. The frame rate looks really good - on the order of 25 fps I would estimate. And the environment you are using doesn't seem to be particularly information-rich. Stereo or mono?
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

It's my stereo camera setup, but the demo is sort dual-mono. I was doing mono-tracking on each camera and fusing the data together to increase the effective FOV.

About the FPS, one of the reasons I want to rip everything apart and start from scratch is to more easily optimize. I can say that it feels pretty responsive when wearing the HMD, however.
Krenzo
Binocular Vision CONFIRMED!
Posts: 265
Joined: Tue Sep 07, 2010 10:46 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Krenzo »

I demoed Valve's system, and yes, they have markers on the walls to overcome the blank wall problem. However, they acknowledged that a successful optical tracking system needs to solve this problem in a less intrusive manner.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Yeah, it probably would have to end up being cleverly designed posters or something.
geekmaster
Petrif-Eyed
Posts: 2708
Joined: Sat Sep 01, 2012 10:47 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by geekmaster »

FingerFlinger wrote:Yeah, it probably would have to end up being cleverly designed posters or something.
Or wallpaper with hidden fiduciary patterns?
User avatar
Chriky
Binocular Vision CONFIRMED!
Posts: 228
Joined: Fri Jan 27, 2012 11:24 am

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Chriky »

Others have mentioned projecting a pattern with a laser machine like you get in clubs. That could guarantee an interesting surface.

Good to see you still plugging away. I've recently done a little bit more work on similar stuff, hopefully post something soon.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by WiredEarp »

This is cool - I suspect the inside out type of tracking will eventually be more popular than the currently popular outside in style.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by zalo »

This is super cool!

I wonder if gravity-compensated accelerometer data can be averaged with the Optical Flow data for slightly better dead reckoning... but they'd have to be calibrated together precisely for there to be any benefit...

Judging by the planned devkit prototypes, they'll have front facing stereo cams so everyone will get to try out your demos!

An existing TrackIR system might be useful for doing intermittent corrections/calibrations since they can do 6 dof across a narrow angle too. Perhaps my efforts might also be useful in that regard (for HMDs with orientation trackers). I suspect intermittent recalibration will offer a wider/freer playing space than relying on absolute systems alone (though one of those would be nice too).
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

What I've found that you can do with the accelerometer is compute a running average of the readings to converge on the user's "neutral" pose. Then, when the user is near the neutral pose, you bias the dead reckoning offset back towards (0, 0, 0).

It technically works (with regards to combating drift), but... you would need to spend more time on the idea to get anything that somebody would actually want to use. My quick version did NOT feel good with the HMD on.

And I think that the premise of the user having a "neutral" pose is suspect in VR, since the head is now an input device.

EDIT: Yeah, once I get further along, I'm definitely going to want to use an outside-in system to measure error, and possibly do some predictive modeling with least-squares optimizations and whatnot. I think predictive modeling could be a HUGE boon to visual odometry for this application, but I know that it will be a challenge. Definitely not at the top of the todo list right now.

EDIT2: There is some REALLY fun stuff you can do for computer vision when you have a highly accurate, low-latency IMU available! But I won't reveal all of my secrets yet.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Image
Image

Since I will be playing with some new mounting configurations for my cameras, I decided to finally automate the painful stereo calibration process. I spent tonight updating a couple of my tools for the PS3 Eye camera: a stereo calibration tool, and a tool to record imagery using the the calibration matrices.
GitHub link
Binaries
Note: The binaries depend on CLEyeMulticam.dll, which I'm not allowed to distribute, but you can get it from Code Laboratories

The first image above is the uncalibrated depth map, and the second is calibrated and stereo rectified. It wasn't a particularly good calibration, but you can see that the second depth map has a few points that the first is missing, so that's good. And now that the process is much quicker, I can spend some time trying to minimize the reprojection error, which is critical for stereo vision.

The depth maps are a new feature in the demo of my sparse stereo-matching library. I haven't pushed those changes to GitHub yet, but they are coming, along with a few options for different neighborhood sizes.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by brantlew »

Good stuff. Camera calibration is definitely a pain. One tip that I've learned for stereo calibration is to separately calibrate each camera and then feed that into rectification, instead of trying to do the all-in-one stereo camera/rectification procedure. Reason being is that when you film the grid pattern in stereo you can't get full coverage across the entire field of view of each camera because you have to find compromise positions in the center where the grid is fully visible to both cameras. Calibrating separately allows you to cover the entire range of each camera.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

That's a good idea, I'll try that!
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

I've been toying around with some ideas over the past week, and I decided that I want to spend some more time on improving my matching library before beginning principle work on the visual odometry portion.

I'll also be expanding this library to include optical flow capabilities because I want to remove my dependencies on OpenCV.

So, my focus in the short-term will be to improve the matching accuracy under stereo and optical flow conditions. I need to do a little bit of memory optimization as well, in order to fully take advantage of SSE instructions.

Image

Here's a dense map made using my stereo library as-is, compared with the ground-truth. It's by no means bad, but I know that it can be much better. My focus is still sparse, so the dense maps will never be as good as they could possibly be, but they are still a good sanity check.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Spent the last couple of weeks optimizing the StereoLib and making a plan for the rest of the project. I'm pretty pleased with my progress so far!

Image
This is a familiar image, but now the census transform and stereo correspondence code is highly optimized for sparse features. Running on 1 core of an i7, I was able to cut the run-time down to .63ms for the census transform and .64ms for the stereo matching. If configured for maximum speed, you can get it all done in under a millisecond! I'm really psyched about this result because that means this is 4-5x faster than the fastest alternative that I know about (And I have been reading A LOT of papers lately).

Image
This is the feature tracking/optical flow result. Pretty reasonable accuracy, but I just have a naive implementation right now, so it is dependent on appropriate configuration. It's also reasonably snappy, at 556 feature matches in 2.1ms. Realistically, it should be much faster, but here I used a large displacement between the two images for illustrative purposes. This means that the algorithm had to search a space about 4 times larger than what I anticipate being necessary under video frame rates.

I have not yet implemented dense functionality for either optical-flow or stereo-correspondence; it's something I want to do, but the visual odometry components are more important.

I've decided to just make this all open-source, because there are a lot of great ideas out there in academia, but very little work is being published about actually bringing the best bits together into a state-of-the-art solution. Hence, I made a new repo on my GitHub page to host the project.

The plan is to make a fully-featured and configurable visual odometry library that is cross-platform and standalone C++. Right now, I have completed a (slightly) modified version of the FAST feature tracker, and most of the CorrespondenceLib. There are also placeholders showing the structure of the other components, and several test harnesses/demos. The demos use OpenCV for visualization, but everything else is just C++11. I've verified that it builds for MSVC and GCC, haven't checked Clang yet. Also, I am heavily using intrinsics up to SSE4, and don't have alternatives in place yet; that will be one of the niceties to be done after primary functionality is established.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by brantlew »

Good stuff Finger! I need to pull down the code this week.
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

You may have a look at this recent paper for dense estimation : http://www.disneyresearch.com/project/lightfields/

I didn't read it yet and don't know if that's better than the state-of-the-art algorithms listed on the Middlebury stereo evaluation page, but the video looked quite convincing to me.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

That's highly impressive! It's also much better quality than any stereo reconstruction is ever likely to be.

They are using a plenoptic camera, which can simultaneously capture imagery at various focal lengths and, to a degree, various perspectives. They are also using around 100 captures per reconstruction, and doing calculations offline. Furthermore, they are using global reconstruction techniques such as belief propagation and exploiting scene geometry.

People use global techniques for stereo reconstruction as well, and they are superior in terms of quality, but much slower. For SLAM/visual odometry, it's typical to use feature correspondences and local reconstruction techniques, because those are what is needed for the 8-point problem and registration to a point-cloud. They are also much faster.

I think this stuff is really cool, nonetheless. Keep posting papers if you find something interesting.
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

I really need to read that paper now. :)

Btw, I already tried to implement a simple dense reconstruction algorithm (SAD and equivalent functions) some time ago (See here) and by looking at your teddy bear results I think you should try to restrict your algorithm to a single line for feature correspondence, else you'll get smearing in the vertical axis (visible on the chimney especially).

Since the images are stereoscopic and already rectified, corresponding points should lie on the same (epipolar) line. You can still use a window correspondence with a height bigger than 1 pixel, but point correspondences should only be searched on the same line in the second image.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Yeah, I am already using the epipolar constraint. I believe that the reason my previous dense depth map was so fuzzy is because I was using a sparse correlation window, only sampling 16 points. That works fine for feature correspondence, because you are only examining pixels that are highly distinctive.

To get my dense map, I simply generated a list of "features" for every single pixel in each image, and ran it through my regular sparse matching code. In reality, most of the image is going to be fairly non-distinctive, and therefore, you need to sample certainly more than 16 points to determine a correlation between pixels. I plan to use a dense correlation window around 11x11 pixels for my dense maps this time, which is almost 8 times more work, per pixel! And since it's dense, there are way more correspondences to generate.

So basically, I need to have a fairly optimal algorithm if I want to generate dense maps at real-time. Planning to use the moving window technique, as described here. In concept, it's a pretty simple solution, but with a naive implementation, you'll fill up the cache pretty darn quick, so I want to think about an elegant way to put it together first. I'm also ready to take a break from feature correspondence and move on to some other stuff for a while.

I didn't know that you had already done some work on this stuff, cool! That makes me feel a little better about posting all this abstract computer vision stuff in a VR forum.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Just made a little demo to show the feature-tracking capability of my new project with the PS3 Eye. As I mentioned before, this is a first blush at feature tracking, but using RanSaC for outlier rejection, it might be good enough for me to able to get working on the actual position tracking part of this project.

If you have a PS3 Eye and want to check it out, just plug it in, unzip the files and run. I've exaggerated the flow vectors to make it easier to see the actual motion.

NOTE: Your CPU must have the SSE4 instruction set.
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

FingerFlinger wrote:I didn't know that you had already done some work on this stuff, cool! That makes me feel a little better about posting all this abstract computer vision stuff in a VR forum.
Several other people already tinkered with this in this forum, it's a bit more technically oriented than most other forums fortunately, even though the recent Rift craze had a bad effect on the signal/noise ratio.
FingerFlinger wrote:If you have a PS3 Eye and want to check it out, just plug it in, unzip the files and run. I've exaggerated the flow vectors to make it easier to see the actual motion.
No plan for a Linux version I guess ?
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

Fredz wrote:Several other people already tinkered with this in this forum, it's a bit more technically oriented than most other forums fortunately, even though the recent Rift craze had a bad effect on the signal/noise ratio.
Oh I know that; I've been lurking since 2011, but didn't see those topics. Can you point me to any good threads?
Fredz wrote:No plan for a Linux version I guess ?
Well, the library should work on Linux, but that little demo uses the CLEye SDK, which is Windows-only. I know there are drivers for Linux and Mac, but I haven't played much with them. Everything I put into the library will be fully cross-platform.
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

FingerFlinger wrote:Oh I know that; I've been lurking since 2011, but didn't see those topics. Can you point me to any good threads?
Seems one of them was by you and you made an intervention in the other one (by brantlew), forgot you were the same guy. :P I also forgot the other ones were marker-based, did a quick search...
Fredz wrote:Well, the library should work on Linux, but that little demo uses the CLEye SDK, which is Windows-only. I know there are drivers for Linux and Mac, but I haven't played much with them. Everything I put into the library will be fully cross-platform.
Ah ok, I've been using the PS Eye with the PS Move API on Linux and it worked quite well, so there's still hope I guess.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

The vision libraries just need a char*, but there will be a Camera interface for convenience. I didn't know there was a PSMove API for Linux; I was planning to use the gspca-ov534 driver. Have you used it?
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

Nope I just used the PS Move API, no idea what driver is used for the PS Eye in this case.
User avatar
android78
Certif-Eyable!
Posts: 990
Joined: Sat Dec 22, 2007 3:38 am

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by android78 »

This looks like it's working better then I expected it would. I do wonder if there could be a benefit to having a fixed pattern laser projector that you could just stick on the roof and would project on the walls (and possibly floor) though. I'm thinking that a laser pointing at a rotating series of mirrors set at different angles could be really simple and cheap way for creating a fixed feature set for detection.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

@android87

Are you talking about the optical flow demo? If so, I'm glad it's working well for you. :)

Salting the environment definitely helps in environments that are not very feature-rich. You would want the features to remain immobile however, I'm not exactly sure if that's what you mean by rotating mirrors. A potentially big advantage to using known patterns is that you can reacquire an absolute position when they are within view. If Oculus does end up using stereo vision as part of their head-tracking solution, I anticipate that they might display some kind of fiducial logo on the monitor. This would go a long way towards a stable solution that does not require a separate base station or projector.

I've found that proper (and consistent) lighting is a bigger deal for these PS3 Eye cameras, however. During the evening, I actually need to point an additional lamp at a junk-pile in my room in order to detect a decent number of features. One algorithmic thing that could help further is to address the "clustering" issue that comes with feature detection. The exFAST detector attempts to solve this, and I believe it could improve tracking quality specifically in low-light situations.

EDIT: Somehow I was ignorant to the fact that your laser projector would completely solve my low-light issues! With enough coverage, it would actually be advantageous to have a dark room and only track the actively projected points. I'm still hesitant to require a separate piece of equipment, though. At that point, it seems like you might as well just use an "outside-in" tracker.
User avatar
android78
Certif-Eyable!
Posts: 990
Joined: Sat Dec 22, 2007 3:38 am

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by android78 »

My laser idea would work a lot better if the room is dark. I'm thinking something very simple that could run on a battery like this:
laser pattern generator.jpg
You do not have the required permissions to view the files attached to this post.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

After the discussion with Fredz last week, I was curious to see what dense maps would look like with my new code. It's a naive implementation right now, and takes a few seconds for VGA, but I anticipate real-time speeds once I eventually optimize it.

ImageImage

Same set as the first image, but generated from high-res inputs.
Image

By the way, from what I can tell, that lobe on the chimney in the Teddy Bear set is just an artifact of local correspondence algorithms. I looked at results from a few different papers, and my dense maps are very similar. The lobe disappears at higher resolutions, also.

Dense maps aren't particularly useful for visual odometry/SLAM/ but I have to say I am getting more and more interested in stereo reconstruction. It might be a good area to spend some time when I get burned out on visual odometry stuff.
Last edited by FingerFlinger on Mon Aug 12, 2013 6:55 pm, edited 1 time in total.
JDuncan
Cross Eyed!
Posts: 130
Joined: Wed Feb 09, 2011 3:30 pm
Location: My Left Hand
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by JDuncan »

I saw similar looking pictures when I used avisynth.

http://www.avsforum.com/t/1292065/monst ... t_19541190

I looked at a frame of video, and using multiple copies of the video frame, one video frame had different video quality then the other frame.

Then I compared the two frames using virtualdub and avisynth and any differences between the two frames compared to the original were made visible in a picture that looked like the one you just showed.

Is that what your doing to create such pictures? Are you taking multiple copies of a single image and finding differences to generate that picture?
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by cybereality »

Very cool.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

@JDuncan

No, I am using 2 input images that were captured from slightly different horizontal positions. This means that depth information can be recovered, by the same mechanism as binocular vision. Correspondence Problem
User avatar
Fredz
Petrif-Eyed
Posts: 2255
Joined: Sat Jan 09, 2010 2:06 pm
Location: Perpignan, France
Contact:

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by Fredz »

FingerFlinger wrote:By the way, from what I can tell, that lobe on the chimney in the Teddy Bear set is just an artifact of local correspondence algorithms. I looked at results from a few different papers, and my dense maps are very similar.
Weird, IIRC my tests didn't exhibit such an artifact on the chimney, although I was using a very simple technique. Most algorithms at the Middlebury Stereo Evaluation page don't either. I'll need to find the code on my older machine to test that again.
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Visual Odometry/Position Tracking Worklog [Computer Visi

Post by FingerFlinger »

I think that most of those on the Middlebury eval page are a lot more sophisticated than what I'm attempting. The high-quality local algorithms are using some kind of adaptive weight that acts like a smoothing filter along the scan lines. I've only skimmed a few of the papers. The other factor is that I am using the Census transform, which is significantly different from SAD.

By the way, I assume that your code does straight-forward patch correlation? What window size are you using?

EDIT: This one is from "FPGA Design and Implementation of a Real-Time Stereo Vision System", and is also based on the Census transform. You can see that their results are almost identical to mine.

Image
Post Reply

Return to “VR/AR Research & Development”