There's a pull request from David Ullmann extending the NOPYTHON option.

@David Ullmann (robocyte) Perhaps you can give some insight in regards to performance?

@Brecht Van Lommel (brecht) Good question, to add to sebbas comments, there are a few reasons for the python bindings: a first mundane one is that that's how the mantaflow solver worked originally, and it would have been a lot more work to switch to C/C++ bindings. Also, the performance impact is negligible, as mantaflow provides all the solver building blocks via python, but each of the steps is quite expensive. So there are no low-level operations in python (e.g. access to grid cells), but just a small number of calls to high level functions that typically make several passes over the full grid.

And then the python bindings provide some interesting outlooks: it will make it very easy to merge any future improvements from mantaflow back into blender, and hopefully at some point also allow for very flexible simulations. E.g., users could add own scripted hooks and modifications via python into the simulation loop to change the way the simulation runs without having to recompile. In general, it makes the data flow in the simulations very flexible. In the long run, I think it would also be awesome to have a kind of node editor to set up the simulations, but that's a bit further away :)

Ok. The main concern is not so much that it's slow for fluid simulation by itself, but if multiple performance sensitive areas use Python it does become a problem to have these global locks. If this is how the Mantaflow API works and it's a big burden to use C++ instead it seems acceptable.

For further customization, we should try to figure out a way to fit this into the new node system that is being designed, rather than using Python scripting. We are still trying to work out the design for these kinds of systems, but if there are specific ideas regarding Mantaflow nodes it would be interesting to see what they could look like and how they can fit in.
https://wiki.blender.org/wiki/Source/Nodes

@Brecht Van Lommel (brecht) Okay, yes - the GIL seems to be unproblematic so far. It could make the UI somewhat unresponsive when baking large sims, though. The new node system notes look interesting. I think for mantaflow it would neat to go for the "everything-nodes" direction, where a node graph would define the data flow of the simulation. This could then use a python back-end, or directly hook into the mantaflow functions (the functions that are exposes to python are the crucial ones that ideally would also be available as nodes). It's definitely a good point to keep in mind.

I have another question here for the experienced blender devs (@Brecht Van Lommel (brecht) @Sam Van Hulle (sam_vh) @Martin Felke (scorpion81) @Jacques Lucke (JacquesLucke) and maybe others?): what would be the best next steps for an official 'approval' of the mantaflow diffs? I saw the 2.8 timeline, and I guess it would be ideal to verify that the patch is in a good enough state some time soon. Then it could be merged into the main branch as soon as possible after 2.8 is released in July, such that it can be included for the 2.81 release.

We could also continue with smaller fixes on @Sebastián Barschkis (sebbas) branch on the side. These shouldn't change anything within the larger structure of the diff here, but there are a few smaller cleanup and UI fixes that would be good to address.

The plan is still for @Bastien Montagne (mont29) to do the initial code review. @Bastien Montagne (mont29), will you have time to do this in the next few weeks? Spending like one day to go over the code and testing the functionality.

Yes, it’s still on my radar for this week or next one…

@Brecht Van Lommel (brecht), @Bastien Montagne (mont29) - okay, very good!

@Nils Thuerey (n_t) Hmm, looks like the branch could use a merge with master again. ;)

Some initial usability review after using the branch for a few tests, and a first quick review of the patches:

Results looks great!

Am by no means a specialist of that area, but there is definitively a serious improvement over previous fluid/smoke sims. And I have yet to try all the advanced stuff (especially guiding…).

Usage is less 'user friendly' than with previous sims

• You have to bake to see anything, there is no real-time preview/simulation done during play.
• For fluids, besides the Show FLIP option which can give you an idea of the result, you even need two steps of baking to get a result (first bake the simulation, then bake geometry and/or particles from sim).

I don’t have a very strong opinion on that, it is more a topic to discuss with artists I guess. Yet I’d expect it to make the 'entry point' for beginners/casual users much higher at least… Realize this is also likely a limitation of the python binding design, this could probably not work well in a real-time context?

On the other hand, the ability to separate baking in steps can also be very useful in more production-oriented cases I think, since you save time by not having to recompute everything if you want to e.g. generate finer mesh.

And the possibility to pause and resume baking is also a nice feature.

Showstoppers

IMHO, opened to discussion, and other devs are welcome to shine in on that topic of course.

Cache Handling

If you go to Edit mode, or try to render non-first frame of the (fully baked) simulation, you lose everything and get dummy domain cube again, until you go back to first frame and 'scroll' again… This is a caching bug, which does not affect cloth simulation in same branch e.g., so think there is an issue in how mantaflow integration code handles this?

Incidentally, there is also missing feedback about cached frames in the timeline (the colored band in the cached range of frames).

No Versioning & Crash

There is no versioning done at all. This means that currently old elbeem sim remain, old smoke one is kind of badly converted to new manta (and crashes)… this should be addressed, in worst case by just nuking old sims altogether.

Trying to open a file with some old sims just crashes for me:

Misc

Am getting those messages repeatedly in the console:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 143, in bake_mesh_74
  File "<string>", line 128, in bake_mesh_process_74
NameError: name 'sm74' is not defined

Names are not fully consistent (e.g. the modifier place-holder is still called Smoke, think Fluid would be best here, since it’s the name chosen for the Physics Sims button…). This seems to also sneak in code, where sometimes smoke is used for general MantaFlow handling (in Cycles class name e.g.)…

The old fluid sim code seems to still be around, you can even still use the 'fluid' quick effect operator to set up one, though only half-exposed in the UI then… Think that one should be fully removed.

Adaptive domain is missing (marked as TODO in some areas of code), not a showstopper imho, but still a feature regression.

Code Wise

Some more detailed comments will follow in each differentials, but from general point of view:

• Why try to re-use existing Smoke modifier code? To me it only adds confusion, makes naming fuzzy (prefix vary between defines/enums/structs/functions/etc., from old smoke to (way too) generic fluid to few cases using better manta one…). Some names are even plain old ones from smoke, that is utterly confusing.
• Old Fluid (elbeem) sim code remains half-active, user cannot directly see/edit it but it can still be added indirectly (through quick effect e.g.), this should be cleaned up I think.
• To me it looks like the patches have been split by some kind of logical categorization, but unless am mistaken, they are not independent (as in, I doubt much previous patches can be applied and compile without the last one (D3861)). This is fine for review, but has to be kept in mind for final commit into master (we want all commits there to build and not crash ;) ).

Note: just updated the branch against latest master (was kinda painful, thanks to some 'cleanup' in BKE physics files :( ), would be nice if patches could be updated too.

@Bastien Montagne (mont29) thanks for the detailed comments! We'll have to more carefully look at the points you mentioned. Good idea to make the naming scheme consistent, to properly indicate the parts of the new solver. I'd suggest using "manta" there to distinguish it from the old smoke and liquid version.

And you think it would be better to remove the elbeem and smoke solvers directly? That definitely would be a good next step. The manta code was built using the previous smoke version, hence all the similarities. The old smoke modifier should be removed though. And the liquid one as well, together with all the elbeem code...

The cache handling looks like a bug, I'll try to reproduce that. And regarding the old smoke and liquid sims, I think it would be safest and easiest to add a warning they're not supported anymore. The smoke ones potentially could be converted to some extend, but the liquid sims work quite differently now.

Hi everyone! Some very needed updates on the Mantaflow fluids!

Here are some changes that have been addressed (@Bastien Montagne (mont29) thanks for the detailed comments!)

• The diffs are now ready for a clean merge. I.e. the 12 diffs could be applied one at a time without any compilation issues in the master branch. One only has to consider the new dependency order (1, 2, 3, 10, 4, 7, 9, 11, 6, 5, 8, 12). That is, [Part 2] depends on [Part 1], [Part 3] depends on [Part 2], and so on.
• Old smoke and liquid (Elbeem) code is completely gone.
• Names have been cleaned up. Old smoke references have been replaced with "manta".
• Adaptive smoke domains are now supported too.
• Minor UI cleanups.
• Fix for caching bug.
• Fix for versioning crash

As always, the fluid-mantaflow branch is up-to-date with the diff (and 2.81 of course). Minor fixes will always go there first and it could be merged into the diffs one last time before landing them in master.

@Sebastián Barschkis (sebbas) - neat! I've tested the version in the last days. It compiles and runs nicely on my mac. I think this version is ready to be merged, as far as I can see.

Thanks to adaptive domain this is now feature complete compared to the old system, yay! Thank you for the hard work over all those years.

I discovered two showstopper issues:

• Currently I cannot add modifiers after the manta mod. Many users add modifiers after fluid sims, for example to apply some controllable smoothing that does not require re-bakes.
• The cache indication (colored lines) in the timeline is missing.

A nice to have would be a button that bakes both simulation data and mesh at the same time.

What influence does the parameter for Real World Size have? I have created a demo file with a 10cm domain and a simple drop and the result looks like a huge splash (file is attached):

I found another issue - here the fluid is has an offset to both the inflow and the collision object. The domain has been changed in edit mode:

Hi Gottfried!

The issue with the modifiers should be fixed now! The colored timeline is still a to-do, I'll look into that.

The "Real World size" is currently only being used so that viscosity values can be entered in [m^2 / sec]. Another to-do is to go through all input values and adjust them to world units where possible - taking into consideration the world size (e.g. initial velocities in m/sec).

Your last issue seems to be because of the offset of the origin. If you first apply all transformations and then set the "Origin to Geometry" it should all line up. I see the issue though, better would be to not have to worry about the location of the origin.

Hi everyone,

so during the past weeks I was able to improve some more areas in the fluid-mantaflow branch. I would say these are the final bigger changes before the master merge.

Here's a quick summary:

• Cache: Is much more user-friendly now. In addition to the modular baking system (i.e. bake the base simulation, mesh, and particles all separately), there are 2 new modes:
  • Replay: Similarly to the current smoke cache, which bakes on-the-fly during replay.
  • Final: Which just bakes the current state of the settings with a single button click.
• Fractional Obstacles: Liquid simulations where suffering from stepping effects at inclined obstacles. The new "Fractional" option fixes this problem (it can be thought of as an "anti-aliasing" at the fluid-obstacle interface).
• Code structure:
  • Mantaflow files have moved to extern/ and are clang-formatted (the Mantaflow updater script (also in extern/) handles this).
  • There are only Mantaflow TBB source files now (having the OMP as well gave no real advantage).
• New options: Several parameters, that I feel are very useful for artists, have been added to the UI (e.g. FLIP ratio, timestep min/max).
• Bug fixes: I counted a total of 16 bug fixes since the last diff.

The only feature that's still missing is the cache color indicator in the timeline. I still might be able to add it before the official release though (I am currently working on the patch for that).
The diffs have been updated too (same state as fluid-mantaflow 6b7b619f06af).

So all in all, I would say the branch is ready for master!

Will this by chance bring in OpenVDB integration? According to your website, it stated OpenVDB export (still a bit experimental, though), wasn't sure if that was within the scope of your current branch that you're hoping to get approved into master. And by OpenVDB I presume you mean by the spec of OpenVDB 4.0 so engines like Arnold can directly render them and are directly importable into a DCC application such as Houdini? Also, would this entail any work on some sort of VDB import system to get VDBs imported from Houdini or some other DCC application as well?

Yes, a proper OpenVDB setup where you can import from other DCCs is planned (just not in the 1st release of Manta). For now, the OpenVDB cache option will just save each grid on its own with the name of its parent solver appended (that's why it's "density" and "_s4").

can't wait when blender will be able to import openvdb files from houdini :O once this happens Blender will become a true DCC that u can use in big VFX!

@Sebastián Barschkis (sebbas) +1 for mantaflow to be in master replacing the old system. Important note though, we need to document (in the release notes and the manual) what are the known issues and limitations of mantaflow.

The old system is no longer maintained. People requiring working simulations can always go back to old versions of Blender.

@Dalai Felinto (dfelinto) Okay, great! I will take care of the issues / limitations part.

@Sergey Sharybin (sergey) All patches are now up-to-date. The last two patches D3852 and D3858 have the changes as we discussed them earlier. That is, the renaming from manta to fluid in both DNA structs, RNA and in the Python UI files. Once those two "go green" we can land everything in master.

So the patches are now in the master branch - big thank you to the reviewers!

Here is a short summary of what I plan to do next:

• Move run-time variables out of existing DNA structs. Was mentioned at the end of D3860.
• Update documentation for smoke and write new documentation for the liquid simulation system
• Explain limitations / current issues of the solver in the Wiki. Some of them will also be marked as ToDo for 2.82 and receive higher priority.
• Take care of all the new fluid bugs in the bugtracker