Hazmat Part 3: Choosing the Class

Hey there, it’s Anthony, ready to bring an end to this epic trilogy of identifying hazmat signs.

In this part we will cover how we solved the second half of the generic object detection problem (as detailed in the first part): classification.

The algorithms used in this part are by far the most complicated. Rather than a tutorial on how the algorithms work, this is better described as a tutorial on how to use OpenCV functions that execute the algorithms. Therefore, there will be a lot more code and less explanations than in the first two parts.

First, an algorithm called Scale-Invariant Feature Transform (SIFT) can be used to find “key points” in an image. When applied to a sample image of a hazmat sign, the following output can be found:

It should be noted that SIFT has detected text as features. This enabled us to forego implementing standalone text detection. SIFT has been accurate enough to tell the difference between signs whose only difference is text (like explosives 1.1 and 1.2) surprisingly consistently. The above output was generated by the following code:

An algorithm called k-nearest-neighbors (KNN) can be used to match the similarity of two images based on their features, found by SIFT. The best matching features are chosen, appended to an array called ‘good’ and outputted, as in the following python function:

By feeding the output of that function into the following code:

We can see this output:

The best matching features have been mapped from a template image to a real-world image (of a hazmat sign on my phone).

Using the following function, we can loop over a series of templates and find the one that best matches an inputted real-world image:

In this code, a ‘sign’ class was used to abstract the process of storing information. See ‘classify_abstracted.py’ in SARTVision for the definition of that class, and ‘hazmat.py’ for where the class is used to define a series of templates, which is passed into classify() as the second parameter, sign_list.

The code from this blog post, which is also used by the SART robot, has been taken from the following tutorials, which I strongly recommend as further reading:

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.