Week 4: Time to Test Stuff

February 25, 2016

The Tests

This week I got into the actual nitty-gritty science! I tested the algorithm on different image situations. I ran the algorithm on images with time differences, adjacency differences, rotation differences, and also on 10 images instead of 2. When I say time differences, that just means overlapping images with different shadows. When I say adjacency differences, that means the images overlap from north-to-south instead of east-to-west, because the image pairs I've been using up to this point have all overlapped east-to-west.

The Results

The main data that is of interest from these tests is the number of "good" match points that the algorithm returns. Here is a table of the results:

Image Set	Number of Points
Control	601
Time	63
Rotation (5º)	916
Rotation (45º)	0
North-South #1	68
North-South #2	1
North-South #3	5
North-South #4	0
10 Images	1543

For a more visual representation of these results, here are some images:

Control

Time

5º Rotation

45º Rotation

North-South #1

North-South #2

North-South #3

North-South #4

I didn't include images of the 10 image adjacency because my image outputs only include 2 images. In these pictures, each red dot represents a match point. The matching mechanism used in these examples is called SIFT, or Scale Invariant Feature Transform. In other words, SIFT will match images that differ by size, but not by things like rotation (hence the utter failure of the 45º rotation). If you have questions about why each individual situation performed how it did, I'd be happy to answer in the comments section.

The Problems

Unfortunately, my advisor notified me today that using homographies (see last post) makes tracking errors and bad points messy, so we are going to start using a method of matching features that includes epipolar lines and Essential Matrices. If you're confused, good. I am too. I suppose we'll both find out how Essential Matrices work and how they change our results next week.

The Statistics

Before you start lecturing me about sufficiently large sample sizes, don't worry, I understand that one trial per situation is not up to par in the statistics world. However, if I run the same pair of images through the algorithm more than once, I'll get the same match points every time (trust me I've tried). In addition, the point of these tests was to get a general idea of the strengths and weaknesses of the algorithm in question, not to prove anything in particular. So don't worry, I'll use better statistical methods in the future.