My project requires me to track objects such as cars via a drone.
I have a live video stream in which I would like to detect and track objects, the view doesn't change that much as the drone is nearly stationary moving only about 50 meters left and right at an altitude of 500m. For the proper tracking (and in general better video quality), I would need to stabilise the video using homography. I thought of these steps:
- Align the images to the 1st frame with OpenCV ORB preferable so that the roads and building would be at the same exact location as in the previous frames but moving objects would be not accounted for. This approach is also a bit slow but I can just throw more power at it so it's not that crucial.
- Track the objects using a tracking algorithm.
Unfortunately, when I tried this approach these are the results I got which where unsatisfactory:
Original stock video for testing (the actual drone won't move that much)
Frame 0 and frame 96 matches:
Frame 96 after alignment:
As you can see by the black bar on the right the alignment is completely off, I suspect the moving cars to be the problem. I thought of alining the frame x to the one before it but then the alignment error will build up over time.
Any ideas how I could improve my solution? Maybe some algorithm with knowledge of several past frames?
Small update: My take at first was to eliminate the missalignment problem with a image flow detector like Lucas-Kanade. By detecting the major direction points of movement i could filter out the points that moved in a different direction to the rest. Then using the rest of the points (good points) to find a homography matrix to align the images. However that still wasnt good enough to position the cars. Now I think that I can align the image relative to the satelite imagery. That would eliminate the drifting problem but also introduces many others.
