Can I create a transformation matrix from rotation/translation vectors?

Question

I'm trying to deskew an image that contains an element of known size. Here's a test image:

I can use aruco::estimatePoseBoard which returns rotation and translation vectors. Is there a way to use that information to deskew everything that's in the same plane as the marker board? (Presumably by using the rotation and translation vectors to create a transformation matrix to pass to warpPerspective.)

I know how to deskew the marker board itself. What I want to be able to do is deskew the other things (in this case, the cloud-shaped object) that are in the same plane as the marker board. I'm trying to determine whether or not that's possible and, if so, how to do it.

I can already put four individual markers around the object I want to deskew and use the detected corners along with the known distance between them as input to getPerspectiveTransform. That works very well. But for our real-world application it may be difficult for the user to place markers exactly. It would be much easier if they could place a single marker board in the frame and have the software deskew the other objects.

From the principles of computer graphics (3D transformations) there are methods of creating transformation matrices from such vectors.Btw, skew is not the best way to describe what happen in this image, which seems like a perspective projecton of a rotated plane, to me. — Fat32, Feb 11 '16 at 02:02
@Fat32 No doubt I'm using incorrect terminology and that you're correct about this being a perspective issue, not skewing. So far, I haven't come up with the right search to find a source to teach me how to create the transformation matrix from the vectors. If you can point me to something I'd appreciate it. — SSteve, Feb 11 '16 at 02:10
Computer Graphics_Principles and Practice_2ed_Foley, chs 5 & 6 provide all the necessary mathematics of both 3D transformations and perspective projection issues. Give it a try. Don't expect a solution in 2 days though. It can take more than a week (depending on how many hours you spend) — Fat32, Feb 11 '16 at 03:26
@Fat32 Thanks. I ordered the book. In the meantime I've been reading chapter 10 (Transformations in Two Dimensions) in the 3rd ed which is available free on the book's website and it's already shedding some light. — SSteve, Feb 12 '16 at 12:37
I was actually referring to 2nd edition (from 1990s), but 3rd is also quite fine, at least a modern version. So chapter 10 in 3rd edition refers to 2D transforms. It's one of the most reliable sources. — Fat32, Feb 12 '16 at 22:45

score 0 · Accepted Answer · answered Feb 17 '16 at 01:55

I was stuck on the assumption that the destination points in the call to getPerspectiveTransform had to be the corners of the output image. Once it dawned on me that the destination points could be somewhere within the output image I had my answer.

float boardX = 1240;
float boardY = 1570;
float boardWidth = 1730;
float boardHeight = 1400;

vector<Point2f> destinationCorners;
destinationCorners(Point2f(boardX+boardWidth, boardY));
destinationCorners(Point2f(boardX+boardWidth, boardY+boardHeight));
destinationCorners(Point2f(boardX, boardY+boardHeight));
destinationCorners(Point2f(boardX, boardY));

Mat h = getPerspectiveTransform(detectedCorners, destinationCorners);

Mat bigImage(image.size() * 3, image.type(), Scalar(0, 50, 50));

warpPerspective(image, bigImage, h, bigImage.size());

This fixed the perspective of the board and everything in its plane. (The waviness of the board is due to the fact that the paper wasn't lying flat in the original photo.)

Can I create a transformation matrix from rotation/translation vectors?

1 Answers1