FBX import does not read custom bone properties
Confirmed, NormalTO DO
Actions

Assigned To

None

Authored By

	Charles Wardlaw (kattkieru)
	Sep 5 2019, 6:54 PM

Description

System Information
Operating system: Windows 10
Graphics card: Nvidia 1070 (8gb)

Blender Version
Broken: latest
Worked: Never?

Short description of error
Custom properties on the armature transform export and import fine, but on the bones they are ignored. I spoke with Bastien in blender.chat and he mentioned that this is a TODO that may have fallen through the cracks.

Exact steps for others to reproduce the error
Attempt to export this simple armature (attached), deform bones only, with custom properties. Import it back to see that the properties are not maintained.

export_test_01.blend626 KBDownload

Related Objects

Mentioned In: rBAcb4e5b248c8a: FBX IO: Fix pose bone custom props not being exported.
T68575: FBX Known issues & TODO's
Mentioned Here: rBA2476c0b4b278: FBX IO: Attempt to fix compat of custom props export.
rBAcb4e5b248c8a: FBX IO: Fix pose bone custom props not being exported.
T68575: FBX Known issues & TODO's

Event Timeline

Charles Wardlaw (kattkieru) created this task.Sep 5 2019, 6:54 PM

Germano Cavalcante (mano-wii) assigned this task to Bastien Montagne (mont29).Sep 23 2019, 7:34 PM

Germano Cavalcante (mano-wii) added subscribers: Bastien Montagne (mont29), Germano Cavalcante (mano-wii).

@Bastien Montagne (mont29), is this your area?

Bastien Montagne (mont29) renamed this task from FBX import / export do not read or write custom bone properties to FBX import does not read custom bone properties .Sep 23 2019, 8:49 PM

Bastien Montagne (mont29) removed Bastien Montagne (mont29) as the assignee of this task.

Bastien Montagne (mont29) lowered the priority of this task from 90 to Normal.

Bastien Montagne (mont29) edited projects, added Add-ons (Community), Import/Export; removed BF Blender.

Bastien Montagne (mont29) edited a custom field.

Bastien Montagne (mont29) mentioned this in T68575: FBX Known issues & TODO's.

Bone props are exported, but importer currently ignores them, yes. Added the task to T68575: FBX Known issues & TODO's, thanks.

Charles Wardlaw (kattkieru) added a comment.Oct 4 2019, 11:51 PM

Hi again,

Bone props are not exported, and I think I know why: it's exporting them from the bone and not the PoseBone. As any custom properties used in rigging would be on the PoseBone, it makes it look like nothing is flowing through.

I don't know what use exporting the actual bone properties is; in multiple years of production I haven't see anyone put data in that particular spot.

I ran a test by editing export_fbx_bin.py. I changed line 1517 from:

if scene_data.settings.use_custom_props:
    fbx_data_element_custom_properties(props, bo)

to:

if scene_data.settings.use_custom_props:
    fbx_data_element_custom_properties(props, bo_obj.armature.bdata.pose.bones[bo.name])
)

And low and behold, the props are now exported. However, they're not exported in a place that other apps look for them, near as I can see.

I ran a reference export from Maya and I notice that the "custom" parameters are shoved in with the regular parameters under the node, while the Blender exporter puts them in a Properties70 underneath a NodeAttribute node. I think between the above code change and a slight reorganization of where this data is stored, the bug could be put to bed.

I also noticed that even when the posebone properties are used, the exporter does not look at the min and max values, which is important if they are present in the _RNA_UI.

Attached is a file comparing the reference FBX from Maya (left) and the one I exported from Blender (right), along with that test file.

blender_test_03.fbx11 KBDownload

Charles Wardlaw (kattkieru) added a comment.Oct 4 2019, 11:59 PM

Lemme attach a reference Maya export too, while I'm at it.

two_bone_params_2014.fbx12 KBDownload

Bastien Montagne (mont29) added a comment.Oct 8 2019, 4:22 PM

Thanks, will fix that exporting issue, it's rather trivial.

Bastien Montagne (mont29) mentioned this in rBAcb4e5b248c8a: FBX IO: Fix pose bone custom props not being exported..Oct 8 2019, 4:23 PM

Charles Wardlaw (kattkieru) added a comment.Oct 9 2019, 7:45 PM

Hi there,

The fix you committed in rBAcb4e5b248c8a only fixed half the problem. The other half is the props are being exported in the wrong node, as I indicated in the above image. So, the data is getting written now, but the place where it is being written means that other applications don't pick it up.

Is there a reason why those parameters are being stuck into the extra NodeAttribute chunks that do not appear when exporting from Maya?

Bastien Montagne (mont29) added a comment.Oct 9 2019, 8:27 PM

Did you actually checked my commit? Now we export EDIT bone props in the attribute node (as expected), and POSE bone props in the model (aka 'object') node, that’s what was missing previously. Would not see any reason to write editbone props in the model, that would be only confusing… and would make separation between edit and pose ones impossible.

Charles Wardlaw (kattkieru) added a comment.Oct 10 2019, 5:21 PM

HI Bastien,

I did check your commit, but I'm not sure my image above was understood. Half of the problem was that the data was not getting exported, which it is now thanks to your patch. The other half of the problem is that the FBX file getting exported is malformed.

Please check the image I attached above.

The main issue is now that where the properties are landing in the FBX file is not recognized by other applications, regardless of how they're defined in Blender. The custom pose bone properties need to be showing up in the same property block as all the rest of the regular bone properties, as shown above.

I'm using a slightly modified version of this repo to introspect files:
https://github.com/nem0/OpenFBX

which is how I generated the above image and also am looking to see that data is actually getting exported.

The above file, two_bone_params_2014.fbx, should be used as a reference as it comes from Maya.

If you need any further examples of correct files, please let me know and I can send you anything you like from any version of Maya 2017-2019.

Bastien Montagne (mont29) added a comment.Oct 10 2019, 6:06 PM

You are not addressing my points… again, new code in the FBX exporter puts pose props in the model FBX node, which afaik is the expected behavior…

If I export the example .blend above (F7718563), I get:

["Objects", [], "", [
    ["NodeAttribute", [134216079, "Armature::NodeAttribute", "Null"], "LSS", [
        ["TypeFlags", ["Null"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Color", "ColorRGB", "Color", "", 0.8, 0.8, 0.8], "SSSSDDD", []],
            ["P", ["Size", "double", "Number", "", 100.0], "SSSSD", []],
            ["P", ["Look", "enum", "", "", 1], "SSSSI", []]]]]],
    ["Model", [527288446, "Armature::Model", "Null"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Rotation", "Lcl Rotation", "", "A", -90.00000250447816, 0.0, 0.0], "SSSSDDD", []],
            ["P", ["Lcl Scaling", "Lcl Scaling", "", "A", 100.0, 100.0, 100.0], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []],
            ["P", ["atoms_rabbits", "int", "Integer", "U", 1], "SSSSI", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["Model", [800197842, "root::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Rotation", "Lcl Rotation", "", "A", 90.00000250447816, -0.0, 0.0], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []],
            ["P", ["atoms_rootX", "int", "Integer", "U", 15], "SSSSI", []],
            ["P", ["atoms_name", "KString", "", "U", "bunny"], "SSSSS", []],
            ["P", ["atoms_direction", "Vector", "", "U", 4.0, 5.0, 6.0], "SSSSDDD", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["Model", [317641670, "bone.L::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", 0.08174820244312286, 0.7119274139404297, -0.9106079936027527], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []],
            ["P", ["atoms_position", "Vector", "", "U", 0.08174820244312286, 0.9106079936027527, 0.7119274139404297], "SSSSDDD", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["Model", [217646028, "bone.R::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", -0.5032442212104797, 0.6660205721855164, 0.755639374256134], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []],
            ["P", ["atoms_position", "Vector", "", "U", -0.5032442212104797, -0.755639374256134, 0.6660205721855164], "SSSSDDD", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["Model", [23984158, "center::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", -0.32658660411834717, 1.9007091522216797, -0.22112268209457397], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []],
            ["P", ["atoms_position", "Vector", "", "U", -0.32658660411834717, 0.22112268209457397, 1.9007091522216797], "SSSSDDD", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["NodeAttribute", [858517680, "root::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 3.300000049173832], "SSSSD", []]]]]],
    ["NodeAttribute", [889009370, "bone.L::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 3.300000786781311], "SSSSD", []]]]]],
    ["NodeAttribute", [752467253, "bone.R::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 3.3000020161271095], "SSSSD", []]]]]],
    ["NodeAttribute", [863475931, "center::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 3.3000020161271095], "SSSSD", []]]]]],
    ["NodeAttribute", [489810650, "bone.L_end::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 330.0000786781311], "SSSSD", []]]]]],
    ["Model", [771529387, "bone.L_end::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", 0.0, 1.0, 0.0], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["NodeAttribute", [258117085, "bone.R_end::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 330.00020161271095], "SSSSD", []]]]]],
    ["Model", [818407871, "bone.R_end::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", 0.0, 1.0, 0.0], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]],
    ["NodeAttribute", [785742408, "center_end::NodeAttribute", "LimbNode"], "LSS", [
        ["TypeFlags", ["Skeleton"], "S", []],
        ["Properties70", [], "", [
            ["P", ["Size", "double", "Number", "", 330.00020161271095], "SSSSD", []]]]]],
    ["Model", [573886366, "center_end::Model", "LimbNode"], "LSS", [
        ["Version", [232], "I", []],
        ["Properties70", [], "", [
            ["P", ["Lcl Translation", "Lcl Translation", "", "A", 0.0, 1.0, 0.0], "SSSSDDD", []],
            ["P", ["DefaultAttributeIndex", "int", "Integer", "", 0], "SSSSI", []],
            ["P", ["InheritType", "enum", "", "", 1], "SSSSI", []]]],
        ["MultiLayer", [0], "I", []],
        ["MultiTake", [0], "I", []],
        ["Shading", [true], "C", []],
        ["Culling", ["CullingOff"], "S", []]]]]],

As you can see, all the atoms Pose custom props are in the Model FBX nodes…

Only Edit bones custom props would now be put in the NodeAttribute FBX nodes (don't want to mix those with the Pose custom props on one hand, and on the other, afaik Edit bone custom props are not very commonly used…).

Charles Wardlaw (kattkieru) added a comment.Oct 11 2019, 4:26 PM

Ahh interesting. I think that I might not have had it installed correctly-- a clean build today and I'm seeing the attributes in the correct place! My apologies. But Maya is still not picking them up. [captain picard facepalm gif]

Something else I noticed: the tagging for the parameters is slightly different. At first I thought it is because the props I was exporting aren't integers, but I exported integers from Maya and they do come up differently. Attached are two more images. Could it be that the A+U tag (whatever that means) is what's causing Maya to not recognize the parameters properly?

I'm also attaching one more example file from Maya, a single joint with the six main property types on it, for reference.

export_test_02_maya.fbx17 KBDownload

Bastien Montagne (mont29) added a comment.Oct 11 2019, 6:15 PM

Afaik, A tag means 'animated', and U means 'user-defined data' (this is just assumptions from older FBX I analyzed during main development phase of the add-on, think there are no official info about those anywhere)…

But yeah, maybe customprops all need that mysterious A+U set of tags, FBX has never ever been known for its consistency, so… Will try adding those and we'll see. Thanks for the investigation work, btw! ;)

Charles Wardlaw (kattkieru) added a comment.Oct 11 2019, 8:44 PM

Hey, thank you for listening and helping out!

Bastien Montagne (mont29) added a comment.Oct 12 2019, 1:03 PM

auto-parsing of commits in 2.81 branches is still not fully working it seems… Changed those 'flags' in rBA2476c0b4b2789e65f1ef95989d4e42dfd784be45 (and merged it in master), so this should be testable in next daily build.

Charles Wardlaw (kattkieru) added a comment.Oct 17 2019, 10:36 PM

Just pulled and rebuilt-- the properties are getting through!

Now that this is working in Maya I hope some folks chime in who might want to use it in another package (I hear some people use properties like these to send data into Unity).

Animation on the custom properties is not coming through yet.

As mentioned in the first part of this report the custom properties are not getting read back in if I import the FBX file I just exported. But this is awesome progress!

Bastien Montagne (mont29) edited projects, added Add-ons (BF-Blender); removed Add-ons (Community).Jul 28 2020, 11:00 AM

Bastien Montagne (mont29) moved this task from Backlog to Long Term on the Add-ons (BF-Blender) board.Jul 28 2020, 11:07 AM

Matheus Kirchesch (Kirchesch) added a subscriber: Matheus Kirchesch (Kirchesch).Apr 1 2021, 2:40 AM

UE4 can't import the Custom Properties, does not matter if they are in the edit or pose bones or in the armature level or in the armature object level

There is an Addon called "ue4curves" that does export Custom Properties on the Armature Object Level ("Object Properties" Tab instead of "Object Data" Tab)
But, the armature has to be called root, and any armature that is not called Armature will be imported into UE4 with an Extra Root Bone wich, because aparently UE4 reads the blender Armature Object as a Bone

I have read all the changes this Addon made to the importer and Here they are ("its not much")

File export_fbx_bin

At line 1967

......
                               ACNW(ob_obj.key, 'LCL_ROTATION', force_key, force_sek, rot_deg),
                               ACNW(ob_obj.key, 'LCL_SCALING', force_key, force_sek, scale))
        p_rots[ob_obj] = rot

####### NEW CODE BELOW

for root_obj, root_key in scene_data.data_empties.items():
        CustomCurveObjName = root_obj.name
        CustomCurves = {}
        customACNWnode = AnimationCurveNodeWrapper
        for CustomCurveName in bpy.data.objects[CustomCurveObjName].keys():
            CurveVal = bpy.data.objects[CustomCurveObjName][CustomCurveName]
            if not isinstance(CurveVal, float):
                if isinstance(CurveVal, float):
                    CurveVal = CurveVal + 0.0
                continue
            if isinstance(CurveVal, float):
                CustomCurves[CustomCurveName]=(customACNWnode(root_obj.key, 'CUSTOM', True, force_sek, (CustomCurveName,)), CustomCurveName)

At line 2006

......
for anim_camera, camera in animdata_cameras.values():
            anim_camera.add_keyframe(real_currframe, (camera.lens,))

####### NEW CODE BELOW

if CustomCurves is not False:
            for custom_curve, curve_name in CustomCurves.values():
                custom_curve.add_keyframe(real_currframe, (bpy.data.objects[CustomCurveObjName][curve_name],))

At line 2043

......
for elem_key, group_key, group, fbx_group, fbx_gname in anim_camera.get_final_data(scene, ref_id, force_keep):
            anim_data = animations.setdefault(elem_key, ("dummy_unused_key", {}))
            anim_data[1][fbx_group] = (group_key, group, fbx_gname)

####### NEW CODE BELOW

    for custom_key, (custom_name, custom_value) in CustomCurves.items():
        final_keys = {}
        custom_name.simplify(simplify_fac, bake_step, force_keep)
        if not custom_name:
            continue
        finaldata = custom_name.get_final_data(scene, ref_id, force_keep)
        for elem_key, group_key, group, fbx_group, fbx_gname in finaldata:
            anim_data = animations.setdefault(elem_key, ("dummy_unused_key", {}))
            anim_data[1][fbx_group] = (group_key, group, fbx_gname)

File fbx_utils

At line 731

    kinds = {
        'LCL_TRANSLATION': ("Lcl Translation", "T", ("X", "Y", "Z")),
        'LCL_ROTATION': ("Lcl Rotation", "R", ("X", "Y", "Z")),
        'LCL_SCALING': ("Lcl Scaling", "S", ("X", "Y", "Z")),
        'SHAPE_KEY': ("DeformPercent", "DeformPercent", ("DeformPercent",)),
        'CAMERA_FOCAL': ("FocalLength", "FocalLength", ("FocalLength",)),

####### NEW CODE BELOW

        'CUSTOM': ("Value", "Value", ("Value",)),

At line 741

    def __init__(self, elem_key, kind, force_keying, force_startend_keying, default_values=...):
        self.elem_keys = [elem_key]
        assert(kind in self.kinds)

####### NEW CODE BELOW

        if kind == 'CUSTOM':
            self.fbx_group = [default_values[0]]
            self.fbx_gname = [default_values[0]]
            self.fbx_props = [(default_values[0],)]
            default_values = (0.0,)
        else: ####### THE NEXT THREE LINES EXISTED ALREADY BUT NOW NEED AN EXTRA IDENT FOR THE ELSE AND THE OTHER 3 ARE FOR REFERENCE
            self.fbx_group = [self.kinds[kind][0]]
            self.fbx_gname = [self.kinds[kind][1]]
            self.fbx_props = [self.kinds[kind][2]]
        self.force_keying = force_keying
        self.force_startend_keying = force_startend_keying
        self._keys = []  # (frame, values, write_flags)

He changed this part of your recent fix ( not recent but the one you did based on this post ) At line 586

def _elem_props_flags(animatable, animated, custom):
    # XXX: There are way more flags, see
    #      http://help.autodesk.com/view/FBX/2015/ENU/?guid=__cpp_ref_class_fbx_property_flags_html
    #      Unfortunately, as usual, no doc at all about their 'translation' in actual FBX file format.
    #      Curse you-know-who.
    if animatable:
        if animated:
            if custom:
                return b"A+U"
            return b"A+"
        if custom:
            # Seems that customprops always need those 'flags', see T69554. Go figure...
            return b"A+U"
        return b"A"
    if custom:
        # Seems that customprops always need those 'flags', see T69554. Go figure...
        return b"A+U"
    return b""

To this

def _elem_props_flags(animatable, animated, custom):
    # XXX: There are way more flags, see
    #      http://help.autodesk.com/view/FBX/2015/ENU/?guid=__cpp_ref_class_fbx_property_flags_html
    #      Unfortunately, as usual, no doc at all about their 'translation' in actual FBX file format.
    #      Curse you-know-who.
    if animatable:
        if animated:
            if custom:
                return b"A+U"
            return b"A+"
        if custom:
            return b"AU"
        return b"A"
    if custom:
        return b"U"
    return b""

Maybe he knows something about the types, idk

I don't even know if it helps, but i need to export my custom properties directly to UE4, rn i can do it if i add my drivers to modify my Mesh Shape Keys, but then the animation files get too big and there is a hard cap of -10 and +10 to morph targets or shape keys i forgot blender's name, if you need any help or info i am here to help at any moment

Things i think are happening

Unreal Deletes the Object with its Custom Properties when importing an Armature named "Armature" just so it won't create an extra Root Bone with the Armature's Name

Its late and i forgot my other assumptions, but if anything relevant (in my mind) commes up i will edit the post or create a new one

Brad Clark (RiggingDojo) added a subscriber: Brad Clark (RiggingDojo).Feb 14 2022, 7:00 PM

The UE custom properties stuff is still on my wish list to have fixed, first time finding someone who had posted on it as well.

*Well on everyone using Blender as their only dev tool for UE.

FBX import does not read custom bone properties Confirmed, NormalTO DOActions

Description

Related Objects

Event Timeline

FBX import does not read custom bone properties
Confirmed, NormalTO DO
Actions