Maniphest T92704

Blender crashes when attempting to save the blend file from a script prior to rendering
Closed, Resolved

Assigned To
Jeducious (jameshcrowther)
Authored By
Jeducious (jameshcrowther)
Nov 1 2021, 10:58 AM
Tags
  • BF Blender
Subscribers
Bay Raitt (spiraloid)
Brecht Van Lommel (brecht)
Campbell Barton (campbellbarton)
Jeducious (jameshcrowther)
Raimund Klink (Raimund58)
Tokens
"Doubloon" token, awarded by jameshcrowther.

Description

System Information
System info file for Blender 3.0 Beta tests


System info file for Blender 3.1 alpha tests

Crash Dump


Minimal python script for reproducing the crash

Blender Version
Broken: See above sys info files
Worked: see

Short description of error
When attempting to render using the Crowdrender addon, pressing F12 or using the render still/animation menu options causes Blender to crash. Since I am a developer for Crowdrender, I investigated until I could create a minimum test case for causing the crash. That script is attached above.

The cause appears to be calling the bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile operators whilst in a render_init handler. Further investigation showed that this seems to only occur if I modify a property in the scene or in the bpy.context.window_manager. This combination of modifying such a property, then calling a render, and whilst handling the render_init event, attempting to call bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile appears to be what causes the crash.

I have done more tests and if no such property is modified, then there is no crash, so I am fairly confident this combination is causing the crash.

All of this works fine and has worked fine since Blender 2.83 up to 2.94.

Desired outcome
@Brecht Van Lommel (brecht) has mentioned that what we are doing is the best way to enable our addon to work well with Blender. So my hope is that this problem can be fixed in Blender.

Exact steps for others to reproduce the error

  1. open Blender 3.0 beta or 3.1 alpha
  2. With the default scene loaded, open the scripting workspace
  3. In the scripting workspace, in the text editor, open the script attached to this bug report and click the run script button. It will register a render engine and a handler for render_init events
  4. Press the F12 button, or use the render still menu option
  5. Blender will exit immediately

Revisions and Commits

Closed
rB Blender

Event Timeline

Jeducious (jameshcrowther) created this task.Nov 1 2021, 10:58 AM
Jeducious (jameshcrowther) awarded a token.
Jeducious (jameshcrowther) added a comment.Nov 2 2021, 12:51 AM

Update, have done more testing and its also broken on MacOS as well. Haven't tested linux yet though.

So, on macOS I get the following stack dump

, the sys_info for the machine that created this is .

The problem on macOS is a little different. I have modified my code now so that I do not modify any properties in the scene or window manager. This allows me to render consecutively. The code still saves a copy of the blend file from the bpy.app.handlers.render_init handler I have registered.

On Windows the crash still happens instantly the moment a render is attempted using the crowdrender engine. So modifying properties doesn't appear to be a factor on that platform. However, on MacOS, I can render consecutively, as I said, but if I then modify the scene, I can get Blender to crash when pressing render. This doesn't appear to be consistent, however. There are times when the modification to the scene, and then attempting a render will not cause a crash. The problem is intermittent. I have not done an empirical study yet, so I can't really comment on the percentage of attempts that cause a crash.

Jeducious (jameshcrowther) added a comment.Nov 2 2021, 1:15 AM

Further testing done, I checked the latest python API spec for the bpy.ops.wm.save_as_mainfile operator. Check existing is set to true to warn if the file already exists. To eliminate this as a potential issue I tested setting this to False in my code. Sadly this still causes an instant crash on Windows so this seems to not help.

My thinking was that since warning the user might display a pop-up or write a message to the user, and this is where the code seems to be crashing, that I should perhaps test preventing this information from being displayed. However, my tests showed that setting the check_existing argument of the save_as_mainfile operator did not prevent the crash.

Jeducious (jameshcrowther) added a comment.Nov 3 2021, 12:48 AM

@Brecht Van Lommel (brecht) Finally tracked down the issue and commit that introduced the bug! This issue was caused by a change back in September 17th. https://developer.blender.org/D12507 was created to deal with the following...

When saving blend files close any menus that might be open, show
"waiting" mouse cursor right away, before creating preview.

However this seems to cause a race condition when the bpy.ops.wm.save_as_mainfile or bpy.ops.wm.save_mainfile operators are run from within a handler registered for the bpy.app.handlers.render_init, render_pre or in the render method of the RenderEngine class.

When calling the wm_file_write method in this context, the crash occurs.

I've recompiled Blender and commented out the following from wm_file_write

if (!G.background) {
  /* Redraw to remove menus that might be open. */
  WM_redraw_windows(C);
}

Once recompiled, the crash is now gone, so this looking like the cause of the crash alright.

We've had a chat amongst us devs here at Crowdrender, and have a suggestion, this redraw call is unavoidable when wanting to save the blend file from a script, which will not have any windows open when it does this. So in that case, we'd not want to redraw the UI, no need to. The user is now rendering and probably won't have any UI elements open.

Its also an unexpected side effect of saving a file. I get it that the UI would be altered if the user was saving the file using the UI, but this is a different case.

I think this could be handled better if the redraw call was done from the invoke part of the save operators, rather than the wm_file_write method. Having the redraw call in the code that is writing to the file seems like going against the philosophy of separation of concerns, and in this case is directly responsible for causing a crash.

I am going to experiment by seeing if I can move the redraw call, or achieve the same effect by moving the same call, or something similar to the invoke section of the bpy.ops.wm.save_X operators, since it makes more sense to me that those would be where the user is engaging the save functions of Blender and more appropriate for redrawing the UI.

Be happy to hear your and any other module owners thoughts though..

Jeducious (jameshcrowther) added a comment.Nov 3 2021, 1:08 AM

ok final tests confirmed that I can move the call to redraw to the wm_save_as_mainfile_invoke in wm_files.c.

I recompiled Blender again with the following change to wm_save_as_mainfile_invoke in addition to removing the call to redraw from wm_file_write.

static int wm_save_as_mainfile_invoke(bContext *C, wmOperator *op, const wmEvent *UNUSED(event))
{

  save_set_compress(op);
  save_set_filepath(C, op);

  if (!G.background) {
    /* Redraw to remove menus that might be open. */
    WM_redraw_windows(C);
  }

  WM_event_add_fileselect(C, op);

  return OPERATOR_RUNNING_MODAL;
}

All tested fine in master on windows 10. I was able to save from the UI and run our addon with no crashes and the file still saves in both cases as expected, and there are no glitches in the UI that I found.

Would like to offer these changes as a patch :)

Campbell Barton (campbellbarton) added a subscriber: Campbell Barton (campbellbarton).Nov 3 2021, 3:19 AM

The test script isn't working with factory settings.

Traceback (most recent call last):
  File "/render_engine_script.py", line 68, in before_render
  File "/render_engine_script.py", line 41, in create_temp_directory
AttributeError: 'Scene' object has no attribute 'crowd_render'
Jeducious (jameshcrowther) added a comment.Nov 3 2021, 3:23 AM

@Campbell Barton (campbellbarton) Yeah, thats def on me :( apologies, I missed the fact I had references to our addon in that script. I will fix and re-post the script to remove references to our addon.

Brecht Van Lommel (brecht) added a comment.Nov 3 2021, 4:54 AM

The problem D12507 intended to solve is for the case where you click Save in the menu without opening a file browser. The suggested solution however moves the code to the file browser case, which undoes that fix and also may make the file menu appear in the screenshot.

Instead I suggest to move the call to WM_redraw_windows inside if (BLI_thread_is_main()) { just below, so it will not be called from the render thread.

Jeducious (jameshcrowther) added a comment.Nov 3 2021, 5:15 AM

@Brecht Van Lommel (brecht) I'll make those changes in my branch and report back. Happy to submit a patch like that if it fixes the crash and D12507 still has its intended effect.

Jeducious (jameshcrowther) added a comment.Nov 3 2021, 12:00 PM

@Brecht Van Lommel (brecht), @Campbell Barton (campbellbarton) ok, I've reverted the change in my branch and implemented the suggestion to use if (BLI_thread_is_main()). The testing was a success with no more crashes and the thumbnails/screenshots taken when pressing save do not contain open menus in them, so I think I'll be submitting a branch with these changes as a patch for to close this issue, so long as it passes the review process :P

Brecht Van Lommel (brecht) added a comment.Nov 3 2021, 3:10 PM

Yes, a patch would be welcome.

Raimund Klink (Raimund58) added a subscriber: Raimund Klink (Raimund58).Nov 3 2021, 3:20 PM
Jeducious (jameshcrowther) added a comment.Nov 5 2021, 12:46 AM

@Brecht Van Lommel (brecht) Patch is incoming, I am getting my dev env setup, I was using the GitHub mirror, but after reading the guidelines on the wiki for blender development, seems you guys prefer arcanist? I just installed php and am working on installing arcanist tonight, then hopefully I'll be able to get setup for dev again using the official blender repository rather than the GitHub mirror.

Bay Raitt (spiraloid) added a subscriber: Bay Raitt (spiraloid).Nov 6 2021, 11:15 AM

Hitting this too. so glad to find this thread.

Jeducious (jameshcrowther) added a comment.Nov 7 2021, 3:03 AM

@Bay Raitt (spiraloid) Welcome :) Are you at all able to contribute by submitting a minimum test case for your particular situation? happy to test that on my patch to make sure it actually fixes your case as well :)

Jeducious (jameshcrowther) claimed this task.Nov 8 2021, 1:50 AM

Guess I may as well claim this? I mean, I have created a differential for it now so I think that makes me the person responsible for it xD.

@Brecht Van Lommel (brecht) I've added you as reviewer, should I add anyone else?

Campbell Barton (campbellbarton) added a comment.Nov 8 2021, 2:09 AM

@Jeducious (jameshcrowther) could you update the test script? note that I tried modifying it and couldn't redo the crash in this report.

Jeducious (jameshcrowther) added a comment.Nov 9 2021, 12:46 AM

@Campbell Barton (campbellbarton) I tried, and I failed, urhghghg ๐Ÿ˜ต .

Ok, the test script I wrote appears to have crashed because it was using the same properties as the addon. On its own the test script seems to not crash when Crowdrender is disabled.

So I went with the simpler option of fixing up the addon itself so you can pretty please try testing with that, its attached below, maybe give it a try? I used factory startup and enabled this copy of the addon and I got it to crash. Happy to update the task description too, just thought it might be better to verify this works (um crashes) for you first, otherwise I have more work to do.

To reproduce,
0. On windows 10 64 bit, Start Blender 3.1 alpha or 3.0 beta in factory startup mode (step zero cause I initially missed this step out on the instructions ๐Ÿ˜ ). Default scene is fine

  1. install the addon
  2. enable the addon
  3. Go to render properties panel in Blender and select "crowdrender" as the render engine from the drop down menu where cycles/eevee normally are
  4. A new panel called "Crowdrender" should appear with a single button labelled "START", press start
  5. Press F12 to render the current scene, Blender should crash with an access violation. The crash dump should point to the wm_file_write method where it calls the redraw of the UI.

Campbell Barton (campbellbarton) closed this task as Resolved by committing rBafc60f995701: Fix T92704: Redrawing while saving crashes outside the main thread.Nov 9 2021, 5:22 AM
Campbell Barton (campbellbarton) added a commit: rBafc60f995701: Fix T92704: Redrawing while saving crashes outside the main thread.
Campbell Barton (campbellbarton) added a commit: rBd5d97e416999: Fix T92704: Redrawing while saving crashes outside the main thread.Nov 9 2021, 5:29 AM
Jeducious (jameshcrowther) added a comment.Nov 10 2021, 12:17 AM

Thanks @Campbell Barton (campbellbarton) & @Brecht Van Lommel (brecht) ๐Ÿ˜„ appreciate the help on getting this patch in.

Bay Raitt (spiraloid) added a comment.EditedDec 8 2021, 7:46 AM

@Jeducious (jameshcrowther) man I had to chase this for ages to get a repro. I wrote an addon to do it. this addon loads a template file with a camera in it and tries to save it to a new file. it has something to do with just the presence of a camera in the loaded template scene.

it's not the camera itself (I've rebuilt the camera from scratch) it's just any camera in the loaded scene will crash. scratching my head. I included a blank new scene (no camera) that works just fine. to toggle between them just comment/uncomment the crashme.blend and crashmenot.blend lines (or add a camera to a blank scene as save over the crashme.blend)

such an odd bug. curious if anyone can figure out why this would crash blender 3.0

why would just the presence of a brand new camera in the template scene cause the crash....

(btw I also reset to factory defaults, deactivated all addons, updated startup file. same crash.)

addon installs normally and puts two blend files next to it. comments on the important bits. (you also need to define the filepath variable location to put the new folder in)

Brecht Van Lommel (brecht) added a comment.Dec 8 2021, 2:30 PM

@Bay Raitt (spiraloid), please create a new bug report.

Bay Raitt (spiraloid) added a comment.Dec 10 2021, 9:56 PM