Database image matching

The written word is often what we think about when searching the web. But what if we could search FOR images or even search WITH images. That is what this blog post will address.

But why would you want to search with an image? Because an image says more than a thousand words! You could search for logotypes, markings, signs by simpely taking a picture of them with your phone and getting information without having to write a singel query phrase.

To do this you have to be a mathematical magician or be able to read documentation. Im far from a mathematical magician but I know how to read documentation and comprehend enough to build a imagematching application. For all of this im using a computer vision library called OpenCV, but as OpenCV is a written in c++ and im a .NET developer im using the wrapper library EmguCV.

This example uses EmguCV 3.1.0.1

Preparing the images:

  1. Take all the images from your database and compute descriptors using a DescriptorExtractor. The descriptors can be explained as "interesting" points in the image. There are different algorithms to extract these, in this example im using something called ORB.

  2. Merge your extracted descriptors into a big matrix and save where in this big matrix the individual images descriptors locations are. This is crucial so that you late will be able to find which image is the match.

  3. Build a FLANN index using your merged descriptors. Its important that you use the LshIndexParamses when using ORB to get good results.

Code:

To prepare the images u simply call:

Querying with an image

  1. Find an image you want to use as a Query image.

  2. Extract the descriptors from the query image using the same DescriptorExtractor.

  3. Use KnnSearch on the Flann index to query for you image.

  4. Use the the MappingIndex to find which image the query image is matching.

This project is open source, to get the full source code go to https://github.com/kneeclass/CVImageMatcher/