OpenCv: License Plate Recognition

Question

I have been working on License Plate Recognition based on github repository https://github.com/MicrocontrollersAndMore/OpenCV_3_License_Plate_Recognition_Cpp

but I need to detect small characters. but I can't figure it out. I think I need to change on the size checking but I can't figure it out.

https://github.com/MicrocontrollersAndMore/OpenCV_3_License_Plate_Recognition_Cpp/blob/master/DetectChars.cpp

bool checkIfPossibleChar(PossibleChar &possibleChar) {
        // this function is a 'first pass' that does a rough check on a contour to see if it could be a char,
        // note that we are not (yet) comparing the char to other chars to look for a group
if (possibleChar.boundingRect.area() > MIN_PIXEL_AREA &&
possibleChar.boundingRect.width > MIN_PIXEL_WIDTH && possibleChar.boundingRect.height > MIN_PIXEL_HEIGHT &&
    MIN_ASPECT_RATIO < possibleChar.dblAspectRatio && possibleChar.dblAspectRatio < MAX_ASPECT_RATIO) {
    return(true);
} else {
    return(false);
}}

AND

    double dblDistanceBetweenChars = distanceBetweenChars(possibleChar,     possibleMatchingChar);
    double dblAngleBetweenChars = angleBetweenChars(possibleChar, possibleMatchingChar);
    double dblChangeInArea = (double)abs(possibleMatchingChar.boundingRect.area() - possibleChar.boundingRect.area()) / (double)possibleChar.boundingRect.area();
    double dblChangeInWidth = (double)abs(possibleMatchingChar.boundingRect.width - possibleChar.boundingRect.width) / (double)possibleChar.boundingRect.width;
    double dblChangeInHeight = (double)abs(possibleMatchingChar.boundingRect.height - possibleChar.boundingRect.height) / (double)possibleChar.boundingRect.height;

            // check if chars match
    if (dblDistanceBetweenChars < (possibleChar.dblDiagonalSize * MAX_DIAG_SIZE_MULTIPLE_AWAY) &&
        dblAngleBetweenChars < MAX_ANGLE_BETWEEN_CHARS &&
        dblChangeInArea < MAX_CHANGE_IN_AREA &&
        dblChangeInWidth < MAX_CHANGE_IN_WIDTH &&
        dblChangeInHeight < MAX_CHANGE_IN_HEIGHT) {
        vectorOfMatchingChars.push_back(possibleMatchingChar);      // if the chars are a match, add the current char to vector of matching chars
    }

Thanks a lot in Advance.

Answer 1

You should first debug to see at which conditions the two A, A fails.

1. The MIN_PIXEL_AREA, MIN_PIXEL_WIDTH & MIN_PIXEL_HEIGHT may not be able to accomodate the small size A.

2. In second code snippet you provided, change the syntax of if statement from if(condition1 && cond2 &&...) to syntax if(condition1) {if(codition2) {....}}. This will tell you where these condition fails.

3. Finally, in second snippet, a lot of conditions to check if the bounding rect is a character depends a lot on what kind of character is seen in past. Since in your case, the character AA differs in size, distance and direction(vertical) as well. Thus it would be better to reinitialize for AA instead of using previous characters, or some more conditions should be added for validating characters.[Like if both height and width decreased]

Once you know which conditions fails in step 2 and why, making relevant changes of step 3 should be simple.

---

Edit: I looked further through the repo, and checked function findVectorOfVectorsOfMatchingChars and findVectorOfMatchingChars.

Analysis of findVectorOfMatchingChars function: This function takes a possibleChar and checks if this char is close(when all if condition passes) match with any of the possibleChar of vectorOfChars. If there is a match, store all matches together and return the results

Analysis of findVectorOfVectorsOfMatchingChars function: This function picks any possibleChar from vectorOfPossibleChars and finds all it matches using findVectorOfMatchingChars. If a good match is found, this function calls itself using (vectorOfPossibleChars - matchedPossibleChars).

Now, here is the problem.

Lets say each possibleChar is a vertex of the graph G and there is an edge between two possibleChar iff they satisfy the condition defined in findVectorOfMatchingChars function.

Now, lets say we have a graph with A,B,C,D,X as possibleChar vertex with X close enough to A,B,C,D but A,B,C,D are just far enough of each other to not be considered a close match.

Now let's apply findVectorOfVectorsOfMatchingChars on this vector of possibleChars.

Option 1: If we choose X first, we find A,B,C,D as its matching possibleChar and thus we get all possibleChar.

Option 2: If we choose A first, we find X to be matching possibleChar of A, but not B,C,D. Thus we remove A,X from vectorOfPossibleChars and reapply findVectorOfVectorsOfMatchingChars on B,C,D. Now, since there is no match between B,C,D, we end up with no match for B, or C or D.

Solution to rectify:

1. Create a graph class and register each possibleChar in it as Vertex. Make edges between each pair of vertex using conditions defined in findVectorOfMatchingChars.
2. You may need to customize conditions to incorporate the edges between other vertices and the 2 A's vertex. For this, you should use more datasets, so that the condtion you create or changing of threshold is not too generic to accomodate non-license plate chars.
3. Find connected tree in the graph to find all the characters. This may add all possibleChars. TO avoid that, you can limit addition using weighted edge.