Automatic number plate detection

ALPR stands for automatic license plate recognition. The process typically consists of:

• Identifying potential license plate candidates from an image.
• Ranking the potential plate candidates and selecting the most likely one.
•  Identifying alphanumeric characters and converting them to text.

The hardest part of this process is reliably acquiring the plate candidate image with variables such as:

• Lighting conditions.
• Camera angle.
• Image noise
• Variations in the type of plates.
• Distance to plate; and consequently, the size of the plate in the image

In this article, I'll be going through a gist that works for a few images. I'll cover the literature review in a followup article.

Please note that this implementation has been developed and tested with very few samples; overfitting will probably reduce it's effectiveness.

1. Load an image using opencv and python (Line 33)
2. Run mean shift segmentation (I haven't evaluated the effectiveness of this) and save the image.
3. Use the segmented image as the camera image.
4. Convert the camera image (defaults as a numpy array) into OpenCV Mat and an IPL image (CV2 can work with numpy arrays. Plain old cv requires an IPL or Mat object).
5. Convert the camera image into grayscale.
6. Conduct thresholding on the image (Line 76). This means that pixels that are above the threshold become white and below black. This process is also called binarization.
7. Create a mask that will be used for blob extraction that covers the entire image; setting all the pixels to 1 means that the mask will be applied everywhere.
8. Invert the binarized image. License plates are mostly white with a few characters in black. Inversion means that the plate becomes mostly black with a few white character strokes (Line 94).
9. Blob detection parameters. We don't want very small or very large blobs.
10. Do the actual blob detection (Line 115).
11. Iterate over the grayscale image adding an RGB color << this is a bug
12. Mark potential blobs as black in a white image (Line 131).
13. Another threshold step << This is probably unnecessary.
14. Dilation and erosion to clean up the image << probably unnecessary.
15. Find contours over the black images.
16. For each contour, try and estimate it's overall shape. triangle? rectangle? n-gon (Line 196)? after eliminating very small and very large contours.
17. Draw a bounding box.
18. Calculate an estimate of the aspect ratio. Plates have a W/H ratio of 1 to 5
19. Off the cuff heuristic. Largest rectangle that has an acceptable aspect ratio and is a rectangle is probably a plate. This is not always true. You might much a window or a large rectangular object...
20. For the selected plate candidate, draw a bounding rect

Here are the results:

Segmented image

Thresholded image

Grayscale conversion

Blobs on a white image

Effects of erosion on an image

Effects of dilation on an image

Plate candidates - from contours

Min area box over plate candidate

Bounding box over plate candidate

And here are more images from a few more tests in picassa:

• Failures
• Successes