License Plate Recognition
Project idea
In this project, we will apply the deep learning techniques to solve the problem of license plate recognition (LPR), i.e., given an image of the licensed plate, we want the machine learning system returns us a string of characters and numbers. The overall steps for the task can be summarized as the following several steps[1,2,3]:
- Step1: A sensing system will capture images of the moving vehicles
- Step2: Denoising and super-resolution techniques will be used to obtain images with high quality
- Step3: The computational cost is reduced by applying Segmentation techniques to separate the plate from the whole image
- Step4: Each character and number on the plate will be segemented
- Step5: A decision system will be trained to decide what it finds on the plate, and report the results
- Step6: The trained system will be tested over new data samples
- Step7: We finally evaluate the performance of the techniques we use
Due to the time limit of the course and the project, it’s unlikely for us to finish all the above steps on our own. Instead, we will focus on step 4-7, and write our codes to implement the idea. For step 1-3, we will use the one of following substitutes:
1) directly use a dataset where every image has gone through the step 1-3.
2) generate the data which has gone through step 1-3. In this case, the useful resources include the MLBLR Random Number Plate Generation System and Chinese license plate generation system.
3) collect the data that has gone through and construct our own dataset. In this case, one of the useful resources is Olav’s License Plate Pictures4, which has the images with only the plate. All we need to do is just collecting the data and make the annotations.
use the existing libraries such as tensorflow, opencv, and keras to finish step 1-3. In this case, some potential datasets can be UFPR-ALPR dataset.
Based on our goal, our main tasks for the project (step 4-7) will consist of the following subtasks:
- Task 1: data collection and data pre-processing
- Task 2: character and digits segmentation;
- Task 3: recognition system design, training, and testing
- Task 4: reports writing, including proposal, progressive report, and final report
- Optional Task: If we have time, we will build a mySQL database to match results from our recognition system
Due to the time limit, for Task 1, we will use other people’s work, including but not limited to algorithms and codes. For Task 2, we will still make use of other people’s work but with appropriate modifications and improvements. For Task 3, we will developing the whole system by our own with help only from the libraries introduced in the course.
Dataset datails
We will use datasets from Plate Recognize. It contains several number plate datasets. We will choose datasets here for our project.
Sample from UFPR-ALPR Dataset
This dataset, called UFPR-ALPR dataset, includes 4,500 fully annotated images (over 30,000 LP characters) from 150 vehicles in real-world scenarios where both vehicle and camera (inside another vehicle) are moving. We collected 1,500 images with each camera, divided as follows:
- 900 of cars with gray LP;
- 300 of cars with red LP;
- 300 of motorcycles with gray LP.
Sample from License Plate Detection, Recognition and Automated Storage
The image database contains over 500 images of the rear views of various vehicles (cars, trucks, busses), taken under various lighting conditions (sunny, cloudy, rainy, twilight, night light).
Sample from Medialab LPR database
Software
Paper to read
These papers are included on our plan.
Teammate
Thus the assignments for every member are as follows:
- Jirong Yi: proposal report writing + (Task 1 or Task 2 or Task 3)
- Ke Ma: website designer and modifier + idea of establishing database + (Task 1 or Task 2 or Task 3)
- Qi Qi: idea of car plate detection + finding dataset + polishing + (Task 1 or Task 2 or Task 3)
Progress Milestones
Even though every one of us will focus on certain part of the project, we will get involved in the whole process to make sure that the work we present is up to everyone’s standard. To make the project run as we expect, we set the following milestones:
Before 11/06/2018, we should have finished Task 1 + part of Task 2 + progressive report;
Before 12/04/2018, we should have finished the rest part of Task 2 + Task 3 + final report;
If time permits, we will try to give some theoretical justifications for the whole system and even design new algorithms for segmentation and training.