False SURF output in video

Question

I am trying to find the three cervical vertebrae (C2 , C3 , C4)in x-ray videos (gray images) I am using SURF to identify the place that has these 3 vertebrae (big box for all of them )
SURF gives me good result in this case

Now I am trying to identify each one of these vertebrae from this big box, but SURF failed in this case The SURF works very good with the area that has 3 vertebrae , but not good to find each of them separately

Update : The idea of SURF is finding the important keypoints in the image. therefore, I am using this idea to find the keypoints in 2 images then find the match points between these 2 images. in the beginning of my program , the user select 4 boxes. one big box for all vertebrae and the other 3 of each one.

I will use this selected area as a template to find them in the next images in the videos. first I will apply SURF to find the big box of the all vertebrae. and this work good with this code

now I am tying to find the three small boxes inside the big one but SURF gives me bad results (wrong boxes)

The photo show you the 4 boxes (ignore the left box and the three points ) these are the 4 template images that I am using (after crop it 4 times)

The question , How can I improve the SURF results to get the three vertebrae ?

any help will be so appreciate :D

here is the image that show the perfect results .... big box has 3 boxes ...

enter image description here

This is the code that I am using First parameters (Mat img_object) is the template image that I am trying to find in (Mat img_scene) , this second paramter is the big box that has the 3 vertebrae , third parameter is the size of the box that I wanna draw around the object when we find it

CvRect Identify_SURF_Frame (Mat img_object , Mat img_scene , CvRect in_box) 
    { 
cvNamedWindow("Good Matches & Object detection", CV_WINDOW_AUTOSIZE);
CvRect output_box; 
//-- Step 1: Detect the keypoints using SURF Detector
int minHessian = 1;  // I reduce this number so I can have a lot of number for  keypoints 
SurfFeatureDetector detector( minHessian , 2 , 3 , true , true );
std::vector<KeyPoint> keypoints_object, keypoints_scene;
detector.detect( img_object, keypoints_object );
detector.detect( img_scene, keypoints_scene );

//-- Step 2: Calculate descriptors (feature vectors)
SurfDescriptorExtractor extractor;
Mat descriptors_object, descriptors_scene;
extractor.compute( img_object, keypoints_object, descriptors_object );
extractor.compute( img_scene, keypoints_scene, descriptors_scene );

//-- Step 3: Matching descriptor vectors using FLANN matcher
BruteForceMatcher < L2 < float > > matcher;
std::vector< DMatch > matches;
matcher.match( descriptors_object, descriptors_scene, matches );
double max_dist = 0; double min_dist = 100;

//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_object.rows; i++ )
{ 
    double dist = matches[i].distance;
    if( dist < min_dist ) min_dist = dist;
    if( dist > max_dist ) max_dist = dist;
}

//-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist )
std::vector< DMatch > good_matches;
for( int i = 0; i < descriptors_object.rows; i++ )
{ 
    if( matches[i].distance < 4 * min_dist )
    { 
        good_matches.push_back( matches[i]); 
    }
}

Mat img_matches;
drawMatches( img_object, keypoints_object, img_scene, keypoints_scene, good_matches, img_matches, Scalar::all(-1), Scalar::all(-1), 
            vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );

//-- Localize the object
std::vector<Point2f> obj;
std::vector<Point2f> scene;
if (good_matches.size() >= 4)
{
    for( int i = 0; i < good_matches.size(); i++ )
    {
        //-- Get the keypoints from the good matches
        obj.push_back( keypoints_object[ good_matches[i].queryIdx ].pt );
        scene.push_back( keypoints_scene[ good_matches[i].trainIdx ].pt );
    }

    Mat H = findHomography( obj, scene, CV_RANSAC );
    //-- Get the corners from the image_1 ( the object to be "detected" )
    std::vector<Point2f> obj_corners(2);
    obj_corners[0] = cvPoint(0,0); 
    obj_corners[1] = cvPoint( img_object.cols, 0 );
    //obj_corners[2] = cvPoint( img_object.cols, img_object.rows ); 
    //obj_corners[3] = cvPoint( 0, img_object.rows );

    std::vector<Point2f> scene_corners(2);
    perspectiveTransform( obj_corners, scene_corners, H);
    int x1 , x2 , y1 , y2 ;
    x1 = scene_corners[0].x + Point2f( img_object.cols, 0).x ; 
    y1 = scene_corners[0].y + Point2f( img_object.cols, 0).y ; 
    x2 = scene_corners[0].x + Point2f( img_object.cols, 0).x + in_box.width ; 
    y2 = scene_corners[0].y + Point2f( img_object.cols, 0).y + in_box.height ; 

    rectangle(img_matches , cvPoint(x1, y1) , cvPoint(x2, y2)  , Scalar( 255, 255, 255), 1 );
    output_box.x = x1 - in_box.width ; 
    output_box.y = y1 ; 
    output_box.width = in_box.width ; 
    output_box.height = in_box.height ; 
}
//-- Show detected matches
imshow( "Good Matches & Object detection", img_matches );
return output_box ; 
}

I edit the post to have a photo. for explaining the code, I am using known algorithm called SURF. this will be too long to describe here — seereen, Apr 29 '13 at 18:33
I agree with what Peter said, and while I understand that you don't want/need to describe the entire SURF algorithm, you should give more non-code detail on your implementation method. For instance, rather than simply telling us that 'Mat img_object' is the template, show the template in the post. And try to include some actual results of your current implementation, not just the ideal results. — Sam Maloney, Apr 29 '13 at 19:28

score 4 · Answer 1 · edited Apr 30 '13 at 13:00

4

In brief, what you are trying to do is probably not feasible. Why? Because SURF was not designed to do that.

In a first approximation, SURF is designed as a faster (approximated and with less invariance) SIFT, i.e., a computer vision keypoint detector. Roughly speaking, CV keypoints are either corners or blobs as detected by some kind of difference of Gaussians (the scale-space search is used to auomatically select the best Gaussian widths).

While these keypoints are present in structure-from-motion or urban stereoscopic images, they are clearly absent in your images. If your primary goal is to detect the vertebrae, you should try instead (and for example):

template matching approaches (since you have an accurate visual model of what you are looking for)
or maybe train some cascade or HoG based detector that will be applied using sliding windows on your full images.

edited Apr 30 '13 at 13:00

Libor

4,255
24
38

answered Apr 30 '13 at 06:49

sansuiso

3,937
18
27

why I can do it for the big box not for the small boxes. I think SURF cannot work with small images , right? – seereen Apr 30 '13 at 17:31
the suggestion ideas need to do training, I am trying to find another ideas without training like SURF. I try SIFT and BRISK both of them do not give me any keypoints at all !!! – seereen Apr 30 '13 at 17:38
I have to find something better than template matching :( – seereen Apr 30 '13 at 18:19
BRISK, SIFT, FREAK, etc. are all corner-oriented. You have no corners at all, so template matching is still you best hope. Maybe linear feature extraction to detect the bones ? – sansuiso Apr 30 '13 at 20:05
If I want to use Template matching ... how can I apply the rotation ??? ... the three Cervical rotate little bit in the videos??? – seereen May 06 '13 at 19:37
Strictly speaking, you can't. It will be more difficult. For example, try matching with rotated versions of the templates, or use rotation-invariant feature descriptors. – sansuiso May 06 '13 at 19:55
SURF works with rotation !!! ... but did not give good results for small images ... and the template matching does not provide rotation .. but give good results in non-rotated cases !!! .. so can you give me other option without training :-( – seereen May 06 '13 at 21:07
I rotate the template image to use them in template matching ... now how can I know what is the best match between them ?? – seereen May 08 '13 at 21:40

False SURF output in video

1 Answers1