How is texture baking implemented?

Question

Suppose we already have UV coordinates assigned for mesh vertices, how is texture baking implemented?

I guess it will be something like this:

for each coordinate (u, v) in parameter space:
    (x, y, z) = inverse(u, v) # Get the geometric space coordinate.
    f = faces(x, y, z) # Get the corresponding face. We may need face's normal for rendering.
    pixels[u, v] = render(x, y, z, f)

What is the inverse function? Is it a projective transformation, or a bilinear transformation?

And how to get the corresponding face of one coordinate (u, v) efficiently?

What do you mean by "texture baking"? I'm not familiar with this use of the term. — John Calsbeek, Aug 19 '15 at 14:23
@Calsbeek, its turning 3d calculation on the surface back to a texture in 2d for re-using. Pixar has a paper or technical report where they coin the name. Id search for you but its a bit painfull to do these things on the phone while in transit. — joojaa, Aug 19 '15 at 14:39

score 6 · Accepted Answer · answered Aug 20 '15 at 05:53

Texture baking can be accomplished by simply rendering the mesh in texture space. In other words, you set up a render target matching the size of your texture, and draw the mesh with a vertex shader that sets the output position to the vertex's UV coordinate (appropriately remapped from [0, 1] UV space to [−1, 1] post-projective space).

The vertex shader can still calculate the world-space position, normals, and whatever else it needs and send those down to the pixel shader. Then the pixel shader can do its thing without even being aware that it's rendering into texture space instead of the usual screen space. Lighting and shading calculations will work as usual as long as the pixel shader inputs are hooked up correctly.

You don't have to do any complicated inversion operation; you just take advantage of the GPU rasterization hardware, which is happy to draw your triangles at any coordinates you choose.

Naturally, this requires that your mesh already has a non-overlapping UV mapping that fits within the unit square, but texture baking doesn't really make sense without that.

joojaa · Answer 2 · 2015-08-19T19:29:59.697

The inverse coordinate is not entirely trivial in all cases. But basically, for a basic triangle mesh with nothing exotic and has non overlapping UV's, you need something like this:

For the UV coordinate you need to find the corresponding UV triangle. (step can be eliminated see below)
Then calculate the barycentric triangle coordinate. Most raytracers have simple implementations for this.
From bary coordinate to 3d its just
```
b1*p1_3d + b2*p2_3d+ b3*p3_3d
```
Where b are barycentric cords from step 2. and p's triangle point vectors

How to accelerate the triangle finding.

You can do a a sweeping search in O(log(n)) time. You can also use a a bsp tree to do much the same. This would produce a O(m*log(n)) algorithm where m is number of samples and n triangles. Sounds good.

Wait, we can do better! What if instead of baking each pixel in order we bake each triangle in order. Then you dont have to go find the triangle. Finding the points in a triangle should be trivial. So you get something more akin to a theoretical O(m) algorithm. Which should be both conceptually easier and at least theoretically faster as you eliminate step 1.

Also in the second case you can forego barycentric calculation if you so wish and use an alternate formulation if you wish.

Image 1: Sweeping one polygon at a time is easier and needs not search for polygon in question.

How is texture baking implemented?

2 Answers2

How to accelerate the triangle finding.