Shape Detection Algorithm for Recognizing those famous Traffic Bollards found in Amsterdam

Implemented in WinForms, without any libraries other than those provided by C#

Final grade: 9.5 out of 10

An Amsterdammertje is the typical red-brown steel traffic bollard that is used to separate the sidewalk from the street in Amsterdam. Amsterdammertje is Dutch for 'little one from Amsterdam'. The bollards have the three Saint Andrew's Crosses from the coat of arms of Amsterdam.

This program detects waist-high bollards on the street and pavement in images that can be sourced from Google Maps. Given this angle, we are looking for bollards/poles that appear to (or actually do) get narrower towards the top of the image. Note that Hagenaars also fit this criterium.

Image can be at most 512x512

Original Image

Preprocessing

• Convert the image to Gray Scale by calculating the average of the colors of that pixel using the following weights:

  byte average = (byte)(pixelColor.R * 0.299f + pixelColor.B * 0.114f + pixelColor.G * 0.587f);

  

• Apply a bilateral filter to the gray scaled image to keep edge strength while getting rid of noise.

  

Edge Detection
Canny edge detection is used to produce an edge map of the previous step, with edges of width 1. This is to facilitate a faster and more accurate Hough Transform. Our implementation is slightly adapted. Instead of using a gaussian blur before applying sobels, we use a bilateral filter with a fixed σ = 80. The t_high argument is for our Canny Edge detection, which is determined by using Otsu’s method on the grayscale input image.

Example of Canny Edge detection with q = 85 and σ = 80

Hough Transform
Potential lines were found with angles in range [0.5π,0.75π] with a step size of 0.5π / 180. This ensures only lines that are similar in direction to the sides of the poles are checked for and thus detected.

The red lines in the image above denote the average of the two lines’ mean x coordinates. The most likely lines within this range are paired up with their closest partners to find both sides of every pole in the image.

Pairs that intersect in a positive y coordinate are discarded as they do not approach each other to the top, which a pole in our chosen context would.

Finding the Horizontal Boundaries of the Pole

• Thicker edges are detected in the original bilaterally filtered gray scale image using 4 Sobel convolution kernels. A threshold based on Otsu’s method with q = 86 is applied to that image, which results in the following image.

• Clean outside of pairs. Pixels outside of the pairs found using the Hough Transform, though with a bit of padding to keep the structure of the previously found edge map, are removed from the image to clean up any remaining noise as seen in the following image.

• Object marking. Objects are marked within the previous step’s result using a flood fill algorithm. The bounding boxes for these found objects are computed and marked, as seen in the following image.

• Object filtering. Objects that are insufficiently tall and have a width/height ratio that is too wide are filtered, resulting in the following image

• Final drawing. The areas intersected by the bounding boxes in the y direction and the paired lines in the x direction are colored in on the original image to show the detected poles as seen in the following image.

A large collection of images grouped by success is available in the archive and are organized as follows

• True positives. Many of the images resulting in true positives had a clear foreground and background, or a relatively large difference in local contrast between the pole and the pavement. They were often pictures from a somewhat high angle, so the background was solely pavement with no angles of buildings to get in the way. Two exceptions were darkpoles with bright backgrounds, resulting in particularly strong local contrast and thus a high likely hood of edges being properly detected.

• False Negatives. Noise is a huge problem. Despite our best efforts (canny edge with Otsu’s method and a bilateral filter) some edges are not strong enough to be detected by our Hough Transform.

• False Positives & True Negatives. Finding false positives was actually a challenge. Images of the Eiffel Tower, Bottles, The Washington Monument and deodorant do not get through the flood fill filtering stage. These have been included in the archive. Although they resemble the shape of street poles, intersections and ratio are different enough that they will not be detected. However, any image with a big amount of complexity lines (e.g. animals, hair, fences, etc.) will overload our program when it is trying to find pairs and flood fill. Most likely resulting in a stack overflow of the recursive call. This can be mitigated by introducing limits as a stop condition.

This assignment is a component of the Image Processing course at Utrecht University

Lecturer: Dr. ir. R.W. Poppe