How to transfer an animation from a tree of empties onto an armature (space conversion)?

Question

I'm writing an importer for a game format and everything is fully functional except for the animations, which are still not affecting the skeleton as intended.

Note that the conversion from node to blender bone format is done with the following code, so the bones are reliable and their original matrices can easily retrieved from the bones:

bone.head = mathutils.Vector((0,0,0))       
bone.tail = mathutils.Vector((1,0,0))
bone.transform(node_bind_matrix_worldspace)

Now the animations, I can correctly visualize them as a tree of parented animated empty objects, but I need to transfer these transformations onto an armature.

empty tree

My assumptions are: - empty transformations are relative to the parent empty (ie. there is no rest pose; the original, unkeyframed transformation is simply overwritten) - bone transformations are relative to the bone's own rest pose - the coordinate axes used by blender bones and my animated empties are different

Which means that to get from the empty object transformations to a bone transformations, I think I have to do two things: - remove the rest pose from the transformations - change the coordinate system (the blender bones point in Y-direction, while the empties point in X-direction)

My keyframes are stored in 4x4 matrices called key_mat. To remove the rest pose from the transformations, I do:

key_mat = key_mat * node_rest_mat.inverted()

(I've also tried the "commute", ie. key_mat = node_rest_mat.inverted() * key_mat)

To change the coordinate system, I do the following and it seems to work on the tree of empties:

matrix_swap = mathutils.Euler((math.radians(90), 0,                     
math.radians(90))).to_matrix().to_4x4().inverted()
#...
key_mat = matrix_swap.inverted() * key_mat * matrix_swap

I have also tried axis_conversion(...) from bpy_extras.io_utils but could not get the same result.

The result has the Y axis pointing forward as intended, and is globally rotated as well. Local animations are preserved, so it should be good.

If I do the steps as described above and put everything together, I'm getting a distorted skeletal animation but not the desired result:

skeletal

I just can't seem to produce code that puts those two together into something that creates equivalent animations on the skeleton. Any ideas on how to correctly transfer the keyframes from these empties onto the bones?

Have a look at answers relating to c3d import If you have a rig I'd suggest using the constraints model outlined by JerryNo. — batFINGER, Sep 13 '17 at 16:10
Thanks, but ideally I want to advoid copying and baking the transformation from the empties (which are just a visualization of the problem) to keep the script running at reasonable speed. There surely must be some way to transform the transformations with matrix math. It's just so complicated haha — HENDRIX, Sep 13 '17 at 16:41
Blender includes an addon called "Motion Capture Tools", it should do that for you, or show you how to do it. — sambler, Sep 14 '17 at 03:51
Will do soon once code is cleaned up a bit and I've tried a few more things. ;) Also thinking about portin this replacement of the old matrix code to python... — HENDRIX, Sep 14 '17 at 12:34
I've successfully ported the improved the new C matrix import code to python! Not sure if it really helps in this particular issue but it might help others. — HENDRIX, Sep 14 '17 at 13:16
All done, see answer below for code if you're interested! :D — HENDRIX, Sep 14 '17 at 16:01

HENDRIX · Accepted Answer · 2017-09-14T16:01:04.917

Now I got it!

Imgur

Bones are calculated using this code. Then the rotations are corrected locally, and finally the bones are rotated into the desired spot as if you rotated the whole skeleton in object mode and applied the rotation.

Overview:

#first create the matrices to correct the transformations
matrix_swap = mathutils.Euler((math.radians(90), 0, math.radians(90))).to_matrix().to_4x4()
rotateback = mathutils.Euler((math.radians(-90), math.radians(-90), 0)).to_matrix().to_4x4()

#create and pose the bone
bind = matrix_swap * bind * matrix_swap.inverted()
tail, roll = mat3_to_vec_roll(bind.to_3x3())
bone.head = bind.to_translation()
bone.tail = tail + bone.head
bone.roll = roll
bone.transform(rotateback)

#bring the keyframe into the right space
#we must make this matrix relative to the rest pose to conform with how blender bones work
key_matrix = fallback_matrix[bone_name].inverted() * key_matrix
key_matrix =  matrix_swap * key_matrix *matrix_swap.inverted()

The complete importer code is available on github.

Below is my original answer:

Not really an ideal solution, but at least progress.

Imgur

This brings the keyframe from relative to the parent space into relative to the bone's own rest pose space:

key_mat = node_rest_mat.inverted() * key_mat

The keyframe would then work on the bone if the bone was oriented like the empty.

So I tried to orient the bones like the empties, ie. give them the same local orientation. I could not get it to work with the following 'reliable' bone creation code that I'd like to use:

 bone.head = mathutils.Vector((0,0,0))       
 bone.tail = mathutils.Vector((1,0,0))
 bone.transform(node_bind_matrix_worldspace)

Probably I was doing wrong matrix math with the bones... Instead I had to resort to this answer. I have used that in previous scripts but discarded it in favor of the above code, as manually calculating head, tail and roll tends to mess up the roll on some bones.

I will keep at it and see if I can do it with a nicely aligned skeleton...

How to transfer an animation from a tree of empties onto an armature (space conversion)?

1 Answers1

Linked