Week 3: I knew we'd get to the pictures eventually.

February 18, 2016

This week we finally got to put it all together!

There are several steps to the image matching process: identifying the features of an image, filtering out the "best" features, repeating the first two steps on the other image(s), performing a match between the two (or more) images, and filtering out the bad match points. I'll walk you through the steps.

Before we start though, let me explain a few of the terms:

• Features: a feature, in general terms, is a piece of information that a computer uses to perform a task. In this case, a feature is a specific point in the image, such as a bright spot, an edge, or the center of a crater, for example.
• ANMS: Adaptive Non-Maximal Suppression is a way of thinning out features so that they are more evenly spaced. This is basically done by finding the radius of empty space around each feature and then picking the n number of spaces with the largest radii. What you're left with is a more uniform distribution of features as you can see from the image below.
• RANSAC: Random Sample Consensus is a rather complicated "outlier detection" method that returns data without any influence from the outliers. I don't understand it well enough to explain it, but if you're interested in learning more you can read this technical Wikipedia article...

First, we extract the image's features, which gives you something like this:

Next, we do the same for the other image:

Then, we perform something called Adaptive Non-Maximal Suppression (ANMS), which cleans up some of the noise. This is done to both images, but here we only see one:

After ANMS, we can match the images:

This is not a good match, because there are many intersecting lines; a good match should ideally have no intersecting lines. In order to remedy this, we apply RANSAC, which is an outlier detector. This gives a final product that looks something like this:

This is a good match because, as you can see, there aren't any intersecting lines. And, if you looked closely, you would be able to see that each line matches identical features in each image.

In order for any of this to work, the images have to be overlapping. Otherwise, you would waste your time extracting a bunch of meaningless features from your images. In addition, the images must have the same reference frame. If one of the images is a perspective image and the other is face-on, there are computations that can be made to translate the perspective image into a face-on reference. This is done through something called a homography, and I think it's super cool. It blows my mind that somebody came up with the math and the equations to do something this complicated. While the math is way beyond me, I think you should definitely read more about it just to see how impressive it is.

This project has exposed me to a lot of complex mathematical concepts, all of which have only furthered my interest in the field of mathematics. While I don't necessarily want to pursue a degree in math, I definitely want to learn more about its nearly infinite applications.