How to recognize vehicle license / number plate (ANPR) from an image?

Question

I have a web site that allows users to upload images of cars and I would like to put a privacy filter in place to detect registration plates on the vehicle and blur them.

The blurring is not a problem but is there a library or component (open source preferred) that will help with finding a licence within a photo?

Caveats;

1. I know nothing is perfect and image recognition of this type will provide false positive and negatives.
2. I appreciate that we could ask the user to select the area to blur and we will do this as well, but the question is specifically about finding that data programmatically; so answers such as 'get a person to check every image' is not helpful.
3. This software method is called 'Automatic Number Plate Recognition' in the UK but I cannot see any implementations of it as libraries.
4. Any language is great although .Net is preferred.

Answer 1

EDIT: I wrote a Python script for this.

As your objective is blurring (for privacy protection), you basically need a high recall detector as a first step. Here's how to go about doing this. The included code hints use OpenCV with Python.

1. Convert to Grayscale.

2. Apply Gaussian Blur.

   img = cv2.imread('input.jpg',1)
   img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
   img_gray = cv2.GaussianBlur(img_gray, (5,5), 0)  
   

Let the input image be the following.

3. Apply Sobel Filter to detect vertical edges.

4. Threshold the resultant image using strict threshold or OTSU's binarization.

   cv2.Sobel(image, -1, 1, 0)
   cv2.threshold() 
   

5. Apply a Morphological Closing operation using suitable structuring element. (I used 16x4 as structuring element)

   se = cv2.getStructuringElement(cv2.MORPH_RECT,(16,4))
   cv2.morphologyEx(image, cv2.MORPH_CLOSE, se)  
   

Resultant Image after Step 5.

6. Find external contours of this image.

   cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) 
   

7. For each contour, find the minAreaRect() bounding it.

8. Select rectangles based on aspect ratio, minimum and maximum area, and angle with the horizontal. (I used 2.2 <= Aspect Ratio <= 8, 500 <= Area <=15000, and angle <= 45 degrees)

All minAreaRect()s are shown in orange and the one which satisfies our criteria is in green.

9. There may be false positives after this step, to filter it, use edge density. Edge Density is defined as the number of white pixels/total number of pixels in a rectangle. Set a threshold for edge density. (I used 0.5)

10. Blur the detected regions.

You can apply other filters you deem suitable to increase recall and precision. The detection can also be trained using HOG+SVM to increase precision.