In T67649#747575, @Jacques Lucke (JacquesLucke) wrote:

Should probably be removed.

Thanks for the information. Well I would actually vote for making it work instead of removing. It is related to this problem. This is the functionality that is missing and fixing that would allow us to create fully usable custom compositor nodes add-ons.

I see, unfortunately I never created new nodes for existing node systems, so I'm not sure if there are other ways to solve this, probably not...
This has low priority at the moment. I do not know how hard it is to fix this. A fix of this would probably be part of some larger feature like D2917.

I see, I would really appreciate this feature, it is quite crucial cause this seems to be the only way to make the socket value update work when working with custom nodes. Without it custom nodes lack an important ability to be the part of the compositor nodetree. Thanks for the cooperation.

Maybe @Campbell Barton (campbellbarton) knows the status of the notifier system a bit better.

I also vote for this feature. I'm designing an addon, and I prefer to have a clean separation between model and view (think MVC), where the model should work without Blender and includes a graph, so I want to represent it with Blender nodes but these are only a facade. What I want to achieve is whenever the user changes a value of a node (internal property, or input socket value, or input socket connection), the model is notified, and could lead to some computations. However it is currently impossible to have this notification. My fallback relies on the application handler depsgraph_update_pre, and this isn't great.

In T67649#757307, @Jonathan Giroux (Koltes) wrote:

My fallback relies on the application handler depsgraph_update_pre, and this isn't great.

Thanks for sharing this tip. It is a good workaround. Did you find any way to read the current socket value? I know we can use default_value property but it doesn't work when the socket is connected to another node. I thought of following the link to that node but the actual value might travel through the whole tree.

The trick comes from studying Sverchok source code :)

That's exactly the point of MVC. The graph is only a representation (V for View) of how your model is organized as a graph. What your model really is, is up to you, and Blender is agnostic of that. "Reading the socket value" depends on the model definition of what are nodes and what they do.

For instance, say you're making a graph of math operations. This example is so simple that it could be implemented only on the view side, but imagine a more complex use case. Whenever you instantiate a node on Blender (say node Add), you'd have the information (through the missing callback that this thread is about) that the user wants to create an Add node *on the model graph*. So in the callback (C for Controller) you'd add a node in the model. In the MVC theory, this model change triggers the update of the view, i.e. creates a node in Blender. Currently, Blender creates the node for you, which isn't great if there are some logics in your model (i.e. Blender only checks for compatible socket types, but you may want to add extra checks). Anyway. Whenever you connect two nodes on Blender, you'd affect connections on your model nodes.

Essentially, the model would be like:

class Node:
    def evaluate(self):
        raise NotImplementedError("!")

class BinaryNode(Node):
    def __init__(self):
        super()
        self.lhs_node = None
        self.rhs_node = None

class AddNode(BinaryNode):
    def evaluate(self):
        if not self.lhs_node or not self.rhs_node:
            raise MissingNodeConnectionError()
        return self.lhs_node.evaluate() + self.rhs_node.evaluate()

Note that there are no NodeLink in the model. Indeed, Blender needs a link representation (e.g. you can select it in the editor), but from the model point of view, there's no need for it, nodes simply have references to other nodes.

In my case, I want the graph to be exportable (JSON file I guess) and runnable both inside or outside Blender, so my model would essentially be a data-only graph, and there are two views, Blender's node editor and the runtime version graph. The downside is that for every node, there are three different classes, so it ends up making a lot of code.

Back to the thread. I'm claiming the above example is a stereotypical use case for Blender's node trees, and I'd like that Blender allows for it by giving callbacks on editor events. In order to keep the current behavior (e.g. creating a NodeLink when the user connected two sockets), the callbacks may be initialized with default functions which just do that, and therefore by setting custom callbacks, the current behavior won't happen by default anymore.

I see. Anyway I'd like to point out that with the socket update callback the current value of the socket (not just the default_value but also the one coming from the connected node) should be available in some way.

If you're talking about the manually set value for a non-connected input socket, it's surprisingly available as the socket instance's default_value (whose doc is Input value used for unconnected socket) for already available sockets for primitive types (NodeSocketFloat, NodeSocketVector, ...). Try that out: change a socket value in the editor, and display its default_value in Python (D.node_groups[...].nodes[...].inputs[...].default_value). For custom sockets, there are properties defined in the respective classes.

As I wrote in the comment above, I'd be interested in any kind of value, not just the non-connected one. I think that a unified interface for this would be helpful or a way to get the current value from link.

Then I already explained how to achieve that. Let's not flood this thread; if you still don't get it, contact me. It's good that Blender is in charge of the view only, the model is somewhat free. If you want to do without separating the model from the view, i.e. your node classes inherited from bpy.types.Node are also responsible for computing stuffs, then you need to traverse the graph by yourself, which will be actually more painful, both to implement and to maintain.

Yea the traversing idea is what I don't like actually :D. What you described feels far too complex for someone who just needs to add a new node type. It would be far more clean to be able to access the socket value directly from the basic API. It'd be more addon-friendly :D, focusing just on the particular functionality without the need to go outside your own node checking the graph state. But maybe I'm not seeing all the advantages and disadvantages of both approaches.

It may sound complex, but actually I believe MVC leads to a cleaner code, easier to implement and to maintain, and therefore saves time overall. It pays off even for only a few node types. You may be afraid if you never experienced it before... well if you only have a few node types, I can be a good opportunity to try, so that you'll grasp the concept faster, and you'll be ready for bigger projects later :)

I understand that MVC leads to a clean code but for some reason I think that programming a node should not cover stuff like analyzing the nodes outside and the whole graph. I'd think of node as a unit with interface (sockets and properties) and logic inside, just like a class in C++ for example. But yes in current API situation you are right about that being the best approach. Thanks for sharing that! It helped me a lot. :-)

In T67649#870542, @Germano Cavalcante (mano-wii) wrote:

I cannot reproduce this with either blender 2.80rc1 or current development versions of Blender through the Custom Nodes template.

Adding an example script tested in 2.83 alpha:

Thanks for the reply!

depsgraph_update is not a workaround, it is the correct solution.

The entire node graph should be thought of as something edited by the user, not what gets evaluated. The actual input value for evaluation, how data flows through the graph and gets evaluated is separate from that. Values may be animated or driven, the compositor may transform the node graph for optimization purposes, the graph may get evaluated on the GPU, etc.

We should either remove the update callback, or define it to be called on updates like linking/unlinking sockets. But for direct or indirect changes to input values, that is not going to happen.

I understand that, either way the two update callback functions in the API are confusing and as you said should be removed if not implemented or redefined. However the inability to get the updated socket value is a drawback when creating a custom node. Traversing the tree manually is quite annoying since many possible things can happen and the original value might be transformed along the way. It feels like the custom nodes via Python API functionality is still very limited.

Yes, the API is limited. There are basically only two supported ways to use it:

• Define custom node groups, that are built from existing nodes
• Create a completely separate node system with own evaluation

Everything else is unsupported basically.

We should either remove the update callback, or define it to be called on updates like linking/unlinking sockets.

If I understand correctly, this is already happening? That is what the original report said:

The update callback is not being called when .... It gets called only when a new link is established/removed with the node's sockets.

So you're essentially saying that the model should ask the view about values? I admit values can be animated so any model depends on Blender's animation features, or one would have to reproduce it to be independent of Blender.

BTW what is the animation model in Blender? If an animated value depends on an other (eg driver), how would Blender knows the order of evaluation? Especially if the driver is a complicated, scripted expression. I'm asking because I'm guessing this is a push model (hence order of evaluation), but it may be a pull model (in this case, order is meaningless).

In T67649#870880, @Brecht Van Lommel (brecht) wrote:

Everything else is unsupported basically.

I see, that's a pity though :D. Just this option to access the actual values in connected sockets would solve many problems and basically would allow the developer to do anything with the input data just using Python script which breaks all the current limitations of the model. Please consider this. Blender is highly adjustable and extensible via scripting which I really love but this is really something that is a big obstacle when doing anything with nodes.

I've updated the descriptions in the API now.

It would be good to support writing custom nodes in existing node systems, and with the new function nodes system under development that will likely happen at some point.

But for now it's really not possible to have it working like this. Remember that a node may be used in multiple instances of a node group in the compositor node graph, or separately for each eye in a stereo render, or it may get executed millions of times on the GPU with different values.

In T67649#870894, @Brecht Van Lommel (brecht) wrote:

and with the new function nodes system under development that will likely happen at some point.

That sounds interesting! Thanks for the feedback!

In T67649#870883, @Jonathan Giroux (Koltes) wrote:

So you're essentially saying that the model should ask the view about values? I admit values can be animated so any model depends on Blender's animation features, or one would have to reproduce it to be independent of Blender.

The model here is the user-editable node graph data structure, and that can be viewed and edited in the node editor. But the evaluation is not part of MVC cycle. Think of it like creating a program binary from user-editable source code, and then running that an arbitrary number of times with different input data. The source code is not affected by running the program.

BTW what is the animation model in Blender? If an animated value depends on an other (eg driver), how would Blender knows the order of evaluation? Especially if the driver is a complicated, scripted expression. I'm asking because I'm guessing this is a push model (hence order of evaluation), but it may be a pull model (in this case, order is meaningless).

See here:
https://wiki.blender.org/wiki/Source/Depsgraph

I can't find any page related to a new node system, can you please give a link?

https://wiki.blender.org/wiki/Source/Nodes

I agree on that. But then socket values which are not linked are constants or come from the animation system, right? I mean, you said a couple of minutes ago that the graph could be evaluated on the GPU, but how is it achievable if at every node we'd had to query CPU-side for values?

Thanks for the links.

The dependency graph ensure things are done in the right order. Animation is evaluated on the CPU, then the render engine or compositor is notified about updated values and re-evaluates things as necessary.

Ha! I think we're talking from the start about getting notified about updated values, but of course our conceptions differ from your knowledge.

So is there such a system? Or do you mean the graph is traversed on Blender side (at every pre-render), so that all values are collected, whatever they come from, then the result of this collection is sent to the GPU which is therefore agnostic of any animated value system?

The dependency graph is a complicated system, there are different types of nodes and different renderers that all work differently.

Unless you need this information to implement something specific, I'd rather not spend more time explaining this.

Indeed I'm on a specific project: a custom renderer :) So I'm interested in all info you can give.

I think these info would benefit to a lot of people. Currently my only source of info is studying Sverchok, and I'm even not sure if the code can be trusted at 100%. I understand that the whole system is complicated. But please, make at least an example about how we should use it. There's a template 'Python > Custom Nodes' which shows how to create the nodes, but doesn't say anything about evaluation. I'm willing to enhance this template with evaluation if you don't have time, but I'm still unconfident about the proper way to do it.

Please check the render engine API docs then:
https://docs.blender.org/api/master/bpy.types.RenderEngine.html

The material or node tree datablock will be marked as updated by the dependency graph, and then you can re-export it. There is no granular information about which nodes or sockets changed.

Ok so just to be sure to understand clearly: are you suggesting to build a runtime model based on the node graph, only at startup and later on when the datablock is updated, and then we can process this runtime model the way we want? And there's no other choice than to entirely rebuild the runtime model from zero, when the dadtablock is updated?

Yes, that's correct.

Great, thanks!

just stumble into this
do not forget that
bpy.msgbus.subscribe_rna
is available to get callback from anything

Thanks @NoobiePoopsie (helloidonthaveanyideaformyusername) for this advice. Have you tested it on the sockets? If this works then it might be a solution indeed! Not sure if the node connections also trigger any value change notifications.

i went a bit enthusiastic too fast. it will trigger an update on change only if the socket is unconnected

Ex:

subscribe_to = bpy.data.node_groups['Geometry Nodes.001'].nodes['Combine RGB'].inputs[0]

def msgbus_callback(*args):
    print("Socket Update Callback!", args)

bpy.msgbus.subscribe_rna(
    key=subscribe_to,
    owner="Identifier",
    args=('arguments',),
    notify=msgbus_callback,
    )

It would have been nice to have more API for sockets, evaluate values when possible, or accurate refresh signals
Sorry for the notification

Thanks again for the experiment!!! :-) Yes, I thought so. The unconnected socket scenario works well but the value propagation through the node graph apparently works in a different way.