Also CCing @Bastien Montagne (mont29) to comment on the parity issue.

OK Allow me to try out the 3.1 and 3.2 builds and get back to you.

As far as following the canonical/conventional method for rigging in Blender. That is the method I use. The controller bones are indeed bones, with custom shapes assigned to them to give the animator a UI. All objects in the rig (which is a mechanical rig i.e. no skins or weights) are constrained to those controller bones using child of constraints etc. However I think in this context that might be a moot point. Especially given that there are no direct hierarchies between object and controller i.e. constraints are used as opposed to hierarchical parenting. Which again makes the need to override the entire linked collection seem very inefficient. With respect however, I don't understand how the system is implemented on the back end.

I will do some test with the new builds and report back.

Right so attempting this with Blender 3.1, there is now an option for Make Library Override Single. This is available exclusively for bones and not empties, so using bone controllers as mentioned above is the assumed preferred workflow.

Running Make Library Override Single on one of the controllers from the Outliner's context menu has no visible effect. Shift + Clicking the object data block (to create a local override) in the object properties panel unexpectedly creates a duplicate of it. As opposed to the override behaviour happening in-place. This duplicate can be then transformed (similar to the result of running the create proxy operation). However, the rig is not effected by the transforms of the new duplicate controller at all. Presumably as the constraints are still pointing at the original controller. Which remains locked and in the position read in from the linked file.

Attempting to run the Make Library Override Hierarchy operation on the full collection unfortunately causes Blender to hang once again. Currently for 15 minutes and counting at the time of writing this.

I will run the same test with 3.2 and follow up shortly.

After repeating the same process, Blender 3.2 had exactly the same behaviour as above.

Both versions eventually completed the Make Library Override Hierarchy operation, after leaving them running for nearly an hour. Unlike with 3.0.1 though, the result of this was a complete duplicate of the entire collection with overrides applied (again not an override "in-place") for both 3.1 and 3.2.

After saving new versions from each respective build. Opening the 3.1 file again caused Blender to hang for 2 minutes 53 seconds before eventually loading it. After which the override collection and controllers were fully functional. Opening the 3.2 file again caused Blender to hang for 2 minutes 7 seconds before loading. Again with the override collection and controllers being fully functional thereafter.

With 2.93 on the other hand, with proxies created for each controller. The saved file loads in 7 seconds!

Based on the above. Surely this needs more serious consideration and work, before taking away the proxy system from users?

Is there a way that you could create a blend-file that shows the issue by redacting any proprietary assets for example by replacing them with other objects?

This way it can be used by developers to profile what it taking so much time and cant point to where developers might want to focus time to speed up the process.

OK could you please send me a private email to adam@renderkick.co.uk so that I can send you a download link?

@Ad Morris (morritzio) Blender being an open project, we keep communication about it and its issues also open and public, please attach here such a file, indeed you can taint it as much as needed to remove any sensitive information. Or even generate a dummy example for this purpose.

From your comments though, I suspect override hierarchy ends up dealing with many thousands of data-blocks in a single override hierarchy, for some reasons... If that is the case, it might be worth reconsidering your rigging pipeline, or we may have to tweak override creation process to deal better with those specific cases... But quite impossible to go further without a good example file to investigate.

OK understood. Here's a version of the file with zero verts for all objects. It has the same object tree as the original so should hopefully enable you to run some tests.

OK, so I am not an expert in rigging (@Demeter Dzadik (Mets) please feel free to share your expertise on this), but am fairly sure this Machine collection is not the expected way of rigging something in Blender. From what I can see, you are essentially not using armatures at all, and are instead using parenting (and some constraints) to 'rig' almost 14 000 objects together! Most of those objects should be replaced by actual bones from a single armature.

Some stats:

• 13847 Objects (len(D.collections['Machine'].all_objects)).
  • 6 Armatures (len([ob for ob in D.collections['Machine'].all_objects if ob.type == 'ARMATURE']))
    • 6 Bones in total (sum([len(ob.pose.bones) for ob in D.collections['Machine'].all_objects if ob.type == 'ARMATURE'])).
  • 6774 Empties (len([ob for ob in D.collections['Machine'].all_objects if ob.type == 'EMPTY'])).
  • 11049 Parented Objects (len([ob for ob in D.collections['Machine'].all_objects if ob.parent != None])).
  • 577 Constraints (sum([len(ob.constraints) for ob in D.collections['Machine'].all_objects])).
  • 38656 Relationships from all those objects to other data-blocks (sum(len(v) for v in D.user_map(subset=D.collections['Machine'].all_objects).values())).

In any case, this situation is definitively not supported by library overrides, this is simply impossible. Most likely, such situation would work fairly well with future dynamic overrides, although there would likely be some other limitations to take into account.

Would it not be more reasonable to leave the proxy system available as a fallback (which handles this so called extreme use case without issue) until said dynamic override system is actually complete then?

I believe you will you find a great many users working with CAD data are regularly required to work with object heirarchies of this nature and it isn't as much of a corner case as you think. Regardless of the character centric workflows you're likely more used encountering in the work of the Blender Studio.

I ran some profiling this morning, unfortunately lost the results before making a screen cap, but 90% of the time is spent in pre/post processing of the last bit of remapping code not moved to new multi-processing code from @Jeroen Bakker (jbakker) (namely, calls to BKE_libblock_relink_ex). So I'm going to address this, hopefully it will bring performances back into somewhat acceptable area (a few minutes and not tens of minutes). But again, such case, based on relationships between thousands of Objects, is not really supported by Blender, and we cannot guarantee anything when it comes to it.

FYI, D14240: Add 'multiple' variant of ID relink function. brings liboverride creation on Machine collection linked from this file from almost 30 minutes to about 45 seconds on my machine.

That sounds much more promising :)

@Bastien Montagne (mont29) do you now consider this resolved?

@Aaron Carlisle (Blendify) I keep this around more as an extreme corner case reference for general optimization investigations... this is not really a bug, and in that sense there is no 'solving' it.