Browse DevX
Sign up for e-mail newsletters from DevX


Go Beyond Keywords! Perform a Visual Image Search  : Page 5

Searching graphics files via keywords can be tedious. Learn how to use image-matching technology to find images by matching shapes, patterns, colors, and textures.




Building the Right Environment to Support AI, Machine Learning and Deep Learning

Step 4: Search
eVe lets you perform visual searches, keyword searches, or a combination of the two. For example, you might start with a text (keyword) search to find images in a certain category, followed by visual queries to refine the search after retrieving an initial set of images (e.g. do a keyword search for "tigers" and then click on an image of a tiger to find examples of the animal tiger).

Search types are known as indexes in the eVe API. When you perform a search, you can specify the relative importance of each of the four basic indexes. The four indexes are Eve.COLOR, Eve.SHAPE, Eve.TEXTURE, and Eve.REGION.

Search by color: This is the easiest type of search to understand. It matches the predominant colors in the source and target images. The more colors the objects in the images have in common, and the greater their similarity to the source image, the higher the similarity score.

Search by shape: This type of search uses the two-dimensional outlines of objects within the image as patterns to match in the target image. The closer the shapes of the objects are, the higher the similarity score.

Search by texture: A texture search identifies unique patterns of light and dark pixels within objects in the image, and then ranks the target images based on their similarity to that texture.

Search by region: Region searches are the most advanced search type. In a region search, eVe models the color surface of the region. This essentially gives a 3D shading representation of each object in the image.

For example, if you had a collection of animal pictures, you could retrieve all orange and black tigers (color), or all elephants (shape), or all longhaired furry animals (texture). When the objects are not well defined from their background, search by region can usually detect patterns and similarities.

To run a search you simply:
Figure 4: From a database of faces, beach scenes and baby shots, eVe selects out the images that most closely resemble the query Image.
  1. use an existing MediaCollection to select a MediaObject as a query image source for the search method or pass the 'dummy" MediaObject created from the user selecting regions of the Segmentation Map
  2. define the SearchParameters
  3. perform the search, iterate through all the SearchResults and output the results (see Figure 4)
Here's an example using a previously analyzed MediaObject as the query image.

String sampleMediaObjectPath = new String("/myMediaCollection/edf/mySample.edf"); sampleMediaObject.loadFrom(sampleMediaObjectPath);

Load the MediaCollection that's most likely to contain search matches.

MediaCollection myMediaCollection = (MediaCollection) Eve.newMediaCollection(); String mediaCollectionPath = new String ("/myMediaCollection/"); myMediaCollection.open (mediaCollectionPath);

You need to retrieve the unique keys of all the MediaObjects in the MediaCollection and store them in long array. The keys identify the MediaObjects within the MediaCollection. Any MediaObject can be uniquely identified by referring to its key and the key of the MediaCollection in which it is stored.

long keys[] = myMediaCollection.getKeys();

Create an instance of the SearchParameters class and customize it to your specifications. The setSearch() method accepts three parameters: an int representing the search type—shape, texture, color, or region; a Boolean value that determines whether to include that type of search in the query, and a weight value—the relative importance of the search type on a scale of 0.0 to 1.0. For example, the following SearchParameters instance uses Color and Region search types, and assigns them equal importance:

SearchParameters mySearchParameters = (SearchParameters) Eve.newSearchParameters(); mySearchParameters.setSearch(Eve.COLOR, true, 0.5); mySearchParameters.setSearch(Eve.REGION, true, 0.5);

You perform the search by passing the MediaCollection.search() method the sample MediaObject and the SearchParameters. The search method returns an array of SearchResults objects.

SearchResults mySearchResults[] = myMediaCollection.search(sampleMediaObject, mySearchParameters);

After obtaining the SearchResults array, you simply need to output the findings. While it would be nice to output the actual images in a GUI interface, this example just prints the filenames of the images that matched the search and their respective similarities to the sample MediaObject.

for (int i = 0; i < mySearchResults.length; i++) { MediaObject temp = myMediaCollection.getMediaObject (mySearchResults[i].getKey()); System.out.println(i + " : " + mySearchResults[0].getKey()); String filename = (String) temp.getProperty("originalFilename"); System.out.println( "Target: " + i + " File: " + filename + " Similarity: " + mySearchResults[i].getSimilarity()); }

Thanks for your registration, follow us on our social networks to keep up-to-date