Updated some new code, I'll be looking to clean up the style and comments as well.

I'm continuing the discussion from D8758 here, as it's really more of a design discussion and less code review.

It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is:

1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it.
2. As a container for the UI for per object / per collection strokes.

I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion:

1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.
2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

I think the important part is not using the modifier stack as a UI container unless the line art is actually evaluated as a modifier.
Especially since we already have the collection for grouping settings for multiple objects, these settings currently in the modifier would fit much better there in my opinion.

Add a new object type for Line Art, which would basically be grease pencil without edit mode This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier.

This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Yes, definitely not trivial. So maybe an easier solution with a similar result would be to just use a regular Gpencil empty.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers

This can still be possible if the UI for Line Art isn't in the modifier stack, Line Art could just be evaluated before grease pencil modifiers.

From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

In T80194#1048778, @Hans Goudey (HooglyBoogly) wrote:

It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is:

1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it.
2. As a container for the UI for per object / per collection strokes.

You missed one thing:

3. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything)

I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion:

1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.

Moving the settings around to individual objects and collections would not really solve anything I think.
We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking.

The easiest way to archive this currently is by using modifiers.

If we go this route then the settings will be scattered all over the UI inside different objects and collections.
So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates.

However after talking to Yiming, this could be a way that we get rid of the scene settings.
So now the line crease and chaining settings for example would be per object or collection.

But I think we should keep the current modifier settings in the modifier.

2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

This would remove the major feature this has over freestyle, the ability to bake and edit the strokes in post.
Having different object type would only make things more cumbersome and tedious.

Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry.
So this does not prevent the artist from drawing things by hand in the same GP object.
You can even bake frame 1-5, edit them, then bake 6-10, and continue from there.

Having a new object type for this would destroy any fast and easy workflow imo.

In T80194#1048812, @Brecht Van Lommel (brecht) wrote:

From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

Correct.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

Yes, it is done this way to not lock up the UI completely. This way we can jump around and scrub the timeline without having to wait for it to finish.
(Or just moving around objects in the scene will not lock up blender)

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

As you noticed it is not perfect by any means. Perhaps we should draft something like this in general for heavy modifiers?

So for modifiers that is not "real time" we can have a job system of sorts.
Take mesh modifiers for example:

1. I have the default cube with two modifiers (array and subdiv)
2. The array modifier duplicates the cube 1000 times (nearly instant, no job required)
3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits.
4. When the subdiv modifier job finishes, the result is displayed.

When the job is active you can pan around and scrub the time line, but I guess we would have to lock up the UI if the user does something the requires the full modifier stack to be evaluated.

In T80194#1048853, @Sebastian Parborg (zeddb) wrote:

3. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything)

Fair enough. I imagined being able to "bake" or "apply" the freestyle, but that is indeed starting to look a lot like a modifier again.

Moving the settings around to individual objects and collections would not really solve anything I think.
We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking.
The easiest way to archive this currently is by using modifiers.

I suppose that the collections / objects could get a gpencil object target property. That sort of inverts the relationship though, and it might not be better that way.

If we go this route then the settings will be scattered all over the UI inside different objects and collections.
So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates.

I'm not sure about this one. There are already settings in every object and collection, so it would also be condensing them.

2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry.
So this does not prevent the artist from drawing things by hand in the same GP object.

Okay, the way you explain this makes it clear that was not a good idea. Thanks.

You can even bake frame 1-5, edit them, then bake 6-10, and continue from there.

Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

In T80194#1048915, @Hans Goudey (HooglyBoogly) wrote:

Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

I think this should be case as well. :)
Perhaps we could keep the "bake all" button though?
So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC.

In T80194#1048787, @Antonio Vazquez (antoniov) wrote:

Add a new object type for Line Art, which would basically be grease pencil without edit mode This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier.

I think this idea is worth exploring, Not necessarily a new object type, but a sub category of Grease Pencil, make it a special flag that differentiate with normal ones, this way we have all the Grease Pencil functions available, but we can then approach the UI representation differently. Internally Line Art can still be a modifier, but for this special kind of object we can show the modifier differently, e.g. in a specific tab, and they can then be hidden in the regular modifier stack.

In T80194#1048812, @Brecht Van Lommel (brecht) wrote:

From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

It is thread-safe, it locks main thread during loading, after that Blender data is not accessed while line art is running. The only thing called from the thread is the tag_update() call, if that is thread-safe then we are good to go.

This is currently the best way that we can run line art without blocking the UI. Because the update isn't called by an operator, it's kinda hard for me to use a Job like fluid/physics simulations do.

In T80194#1048862, @Sebastian Parborg (zeddb) wrote:

So for modifiers that is not "real time" we can have a job system of sorts.
Take mesh modifiers for example:

1. I have the default cube with two modifiers (array and subdiv)
2. The array modifier duplicates the cube 1000 times (nearly instant, no job required)
3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits.
4. When the subdiv modifier job finishes, the result is displayed.

Yes! Job access for slow modifiers should be implemented at some point, so that we don't need that much hassle to asynchronously tag everything and check them. But for now, current "worker thread: might just be as good as we can get.

In T80194#1049012, @Sebastian Parborg (zeddb) wrote:

In T80194#1048915, @Hans Goudey (HooglyBoogly) wrote:

Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

I think this should be case as well. :)
Perhaps we could keep the "bake all" button though?
So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC.

This is technically possible, just bake stroke from certain modifiers, there are some small concerns related to it:

1. You might just want to bake line art on top of the stack, as in some cases user might add another line art after some deformation modifiers.
2. Line Art would need 1 entire evaluation for every frame, doesn't matter how many line art modifiers exist. So if you partially bake, it will run the whole thing once again, it isn't really efficient. You can output to different layers in order to separate them.

In T80194#1048778, @Hans Goudey (HooglyBoogly) wrote:

1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.

This doesn't really apply. Currently you can only set "exclude" and such properties for individual ob/collections, because to calculate occlusion, you need everything in the scene right? or you might want to draw some isolate groups and don't care about the relationships with the rest of the scene. (If this is what you want, at the moment it's not really easy to do so, but possible in line art, but I can see there might be some uses to it, and this when exposed, can be used to make some Mesh->GP quick conversions, might be useful for making assets in animations.)

And for the "transparency bits" thing, it might be too technical for the user, Me and @Sebastian Parborg (zeddb) are talking about if we should expose the "bits" as a list, so the user can add entries and name each one, in the modifier they can select the transparency component they want, but the length would be limited to 8 entries (line art internal limit). But this still looks pretty complicated to be honest.

In my opinion, we can think about the bits like the "bone layer" buttons, so it's not actually that hard to understand. (It's better if we make an icon to show in the toggle button if that one has been configured as viable transparency layer in the material panel, that way it's more clear)

To compare the UI of bone layer and line art transparency selection:

To further explain how Line Art transparency mask works, I made a comparison with Freestyle. The transparency setup in the mesh material is necessary, while calculating occlusion info, the transparency bit is registered every time a triangle occludes a line segment. Thus to save space, line art allow up to 8 bits as transparency info, combinations are also allowed to achieve "multiple layers of glass" effect. To achieve regional selection in freestyle like this, one needs to use composition.

After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review.

There are a few immediate issues with the deferred evaluation code:

• It's not obvious that it is thread safe, that the data structures being accessed can't be modified or removed while line art is reading from them.
• It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows).
• It's not clear that the computation is properly cancelled if the modifier, object or scene is deleted.
• The implementation should not use global variables, state should be stored per depsgraph / evaluated scene or object.
• Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps.

In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own..

Some notes about from testing and looking at the code:

• Modifier properties
  • Thickness: what is the unit? can this be clarified?
  • Pre Sample length, Multiple Levels, and more: it's unclear from description what the effect is for end users, descriptions need to be better.
  • Multiple Levels: unclear from description what the effect is for end user
• Scene properties
  • Line Art panel should not be at the top in the order, but rather next to Grease Pencil or even in the same panel
• Edge only meshes without faces result in line art but are invisible in renders otherwise, Ignore these for line art by default to match renders, maybe an option can be added to use them if needed.

Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/.

For transparency masks. The simple solution would be just to have a single option to indicate a different look behind a transparent object, which may cover the majority of use cases.

For more clever ways to have a different look based on some combination of mask values, it would be good to see the actual use case, it's not clear to me what they are. It's tempting to add it because it's easy to do from a programming point of view, but are there really important use cases beyond the basic case?

In T80194#1051927, @Brecht Van Lommel (brecht) wrote:

After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review.

So if this is the case, I'd like to see if there's any way I can implement a deferred evaluation path for the depsgraph, just like @Sebastian Parborg (zeddb) mentiioned that we can have job accessible in the modifier. Line art isn't gonna be really usable if we evaluate multiple modifiers multiple times, unless we are only using it for extremely simple cases.

• It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows).

Humm that's what I haven't thought of. This surely needs per-depsgraph/per-evaluated scene kind of storage.

• Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps.

Yeah this is a problem.

In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own..

Pleeeeease let us implement the Job system for modifiers. I'd love to take on this!

Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/.

This is due to the current nature of line art which needs shared data and stuff, If we do non-threaded updates, this could be fully contained in modifier/.

I'll look into ther UI description problems. For transparency... I can give it a try, where a simple "transparency" toggle would allow selecting those lines behind glass, and if we need more flexibility, we can check an "advanced" button and do it from there.

I think for this patch, I can make the modifiers self-contained and use no thread for doing the updates (If we are okay with the downsides). I'll keep a branched version on the side so there's still some usability. Meanwhile I'll look into the deferred evaluation stuff.

@Matias Mendiola (mendio) Hi, I may also need some suggestion on the UI stuff here :D. I see linguistic description of some stuff may not align that well with western English speakers.

Thanks!

About UI, I suggest to move LineArt modifier settings into a sub-panel of Layers panel in Grease Pencil data properties.
Basically, Lineart modifier properties are properties of a group of strokes (a line set like in freestyle) created into a specific Grease Pencil Layer.
Re-using Grease Pencil Layers abilities helps a lot to define what line set is supposed to be rendered on top of the others.

Levels are similar to Freestyle QI range. The idea is to define Line Set according to occlusion in order to be able to give a different style to hidden lines of geometry.
Here is an example .blend file.

The idea was to be able re-use already existing grease pencil modifiers to create a style specific to this line set like in Freestyle.
As shown in .blend file attached above, that is possible to set multiple LineArt modifiers to define several line sets and give different styles to them.

Those differences may be due to Lineart modifier settings. But also to modifiers that are added afterwards, if layer restriction of modifier is used.

The fact to re-use grease pencil modifiers thought to modify lines in 3D space does not work with those strokes generated from camera angle and projected onto surface.
Something as basic as stroke thickness is not well controlled by a current thickness modifier.

I think that ideally, current Line Art modifier should become a sub-panel of layers.
We have a properties tab dedicated to 2D postprocessing effects that can not be restricted to layers.

Simplest UI would be to have a properties tab dedicated to Line Style modifiers with modifiers thought to give a satisfying result from Camera Angle.
Modifiers shown in this tab would be restricted to the ones defining active layer.
Each layer would have its own Line Art modifier stack.

But user could be satisfied by current modifiers with current restriction to layer ; at condition that those modifiers are adapted to have a LineArt mode to give satisfying results from Camera Angle.

Updated to include the manual page created for line art.

@YimingWu (NicksBest) Congrats on patch landing ! (pun intended :)

I tried it out a little in my spare time, looks like a really great potential for technical-like presentations, kudos!

I was wondering, are there any plans to make an option for zoom-relative line width? It works great as it is, but in some cases if i zoom out to a scene over-all, the lines almost disappear. Likewise, if you zoom in for a detail shot, lines become a little too thick.

Anyways, great effort, thanks a lot for your addition, waiting for a long time for such a feature.