Maniphest T94127

Cycles/Compositor: The denoiser returns a black image if not all requirements are fulfilled
Confirmed, NormalBUG

Assigned To
None
Authored By
Raimund Klink (Raimund58)
Dec 15 2021, 9:47 PM
Tags
  • BF Blender
  • Render & Cycles
  • Compositing
  • VFX & Video
Subscribers
Brecht Van Lommel (brecht)
Philipp Oeser (lichtwerk)
Raimund Klink (Raimund58)
Ryan Inch (Imaginer)
Sergey Sharybin (sergey)

Description

System Information
Operating system: Default KVM (Proxmox, Linux)
Graphics card: None

Blender Version
Broken: Blender 3.0
Worked: Before Blender 3.0 the frame was just not denoised

Short description of error
If you enable OIDN/Optix denoising without having the needed CPU flags you will get a black frame back.
The desired outcome would be an aborted render with a helpful error.

Exact steps for others to reproduce the error

  1. Set up a default KVM
  2. Enable denoising
  3. Render over the CLI one Frame
  4. It will be black

Further information
The config of a default KVM:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   40 bits physical, 48 bits virtual
CPU(s):                          6
On-line CPU(s) list:             0-5
Thread(s) per core:              1
Core(s) per socket:              6
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      15
Model:                           6
Model name:                      Common KVM processor
Stepping:                        1
CPU MHz:                         3399.998
BogoMIPS:                        6799.99
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       384 KiB
L1i cache:                       384 KiB
L2 cache:                        3 MiB
L3 cache:                        16 MiB
NUMA node0 CPU(s):               0-5
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm rep_good nopl cpuid extd_apicid tsc_known_freq pni cx16 x2apic hypervisor cmp_legacy 3dnowprefetch vmmcal
                                 l

The command that I used to render one frame:

./blender -b --debug --debug-cycles --python-expr 'import _cycles;print(_cycles.with_openimagedenoise);print(_cycles.with_embree);print(_cycles.with_osl)' "your path to the .blend" -f 1









If I set up the KVM to use HOST at least OIDN works
HOST config:

Virtualization:                  AMD-V
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       384 KiB
L1i cache:                       384 KiB
L2 cache:                        3 MiB
L3 cache:                        16 MiB
NUMA node0 CPU(s):               0-5
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx1
                                 6 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bm
                                 i1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr virt_ssbd arat npt nrip_save arch_capabilities

Why it would be nice to have the bug fixed:
We have a renderfarm that consist out of random machines and it is always a problem if random frames are black/not denoised/strange colored/broken/out of order.

Revisions and Commits

rB Blender
D13944

Event Timeline

Raimund Klink (Raimund58) created this task.Dec 15 2021, 9:47 PM
Raimund Klink (Raimund58) edited projects, added BF Blender, Render & Cycles, Compositing; removed Add-ons (Community).Dec 15 2021, 9:50 PM
Raimund Klink (Raimund58) updated the task description.Dec 15 2021, 9:53 PM
Raimund Klink (Raimund58) added a comment.Dec 15 2021, 9:58 PM

Another example:
The black frames are rendered by a default KVM, the normal ones by a HOST KVM

Raimund Klink (Raimund58) updated the task description.Dec 16 2021, 2:16 PM
Raimund Klink (Raimund58) updated the task description.
Raimund Klink (Raimund58) updated the task description.Dec 16 2021, 4:21 PM
Raimund Klink (Raimund58) updated the task description.
Raimund Klink (Raimund58) updated the task description.
Ryan Inch (Imaginer) added a subscriber: Ryan Inch (Imaginer).Dec 19 2021, 9:06 AM
Raimund Klink (Raimund58) added a comment.Dec 19 2021, 9:24 PM

Sometimes other features seem to break as well:




The broken frame was rendered by a "Common KVM processor"

Raimund Klink (Raimund58) added a comment.Dec 19 2021, 9:32 PM

Another Blender 2.93 example:



Philipp Oeser (lichtwerk) changed the task status from Needs Triage to Needs Information from Developers.Jan 24 2022, 12:43 PM
Philipp Oeser (lichtwerk) added a project: VFX & Video.
Philipp Oeser (lichtwerk) added subscribers: Sergey Sharybin (sergey), Brecht Van Lommel (brecht), Philipp Oeser (lichtwerk).

T76668, T94333 could be related.

@Sergey Sharybin (sergey), @Brecht Van Lommel (brecht): I am unable to test this, is this something to be expected?

Raimund Klink (Raimund58) added a comment.Jan 27 2022, 12:36 PM
In T94127#1295825, @Philipp Oeser (lichtwerk) wrote:

Maybe, but we can identify that better after he attaches his system-info.

Sergey Sharybin (sergey) added a comment.Jan 28 2022, 1:30 PM

Proposed fix for the Cycles part is at D13944. Note that SSE 4.1 is required for OIDN, so the patch will effectively make render fail on the configuration where SSE 4.1 support is not detected.

Not sure what is the best way to tackle Compositor node.

For the extra features which are not working reliably (mentioned in T94127#1275555) think is better to submit a separate report with a demo .blend file.

Raimund Klink (Raimund58) added a comment.Jan 28 2022, 2:19 PM

For the extra features which are not working reliably (mentioned in T94127#1275555) think is better to submit a separate report with a demo .blend file.

I would do that if I still had the .blend. But I only have the screenshots. Sorry about that :/
But I think that mainly the denoise broke, but I can not verify that.

Thank you for fixing this <3

Raimund Klink (Raimund58) added a comment.EditedJan 28 2022, 2:32 PM

@Sergey Sharybin (sergey) You wrote:

This change makes it so that if OIDN is not supported on the current
CPU Cycles will report an error and stop rendering. This behavior is
similar to when an OptiX denoiser is requested and there is no OptiX
compatible device available.

But in my tests I also got a black frame from Optix denoising.
Are you sure about the behaviour or did I make a mistake?

Sergey Sharybin (sergey) added a commit: rB430f71fce289: Fix insufficient CPU flags checks for Cycles OIDN.Jan 28 2022, 2:46 PM
Sergey Sharybin (sergey) added a comment.Jan 28 2022, 3:01 PM

@Raimund Klink (Raimund58) What a great question! In my initial implementation that what would've happened. What I didn't realize is that this change has been made rB0ab1b19de4da.

Sergey Sharybin (sergey) added a comment.Jan 28 2022, 3:15 PM

@Raimund Klink (Raimund58) The complicated part is that we kind of want .blend files to be easily shareable between artists in a way that they can render them without running into errors, and this is an opposite of what one'd want for the render farms and studios. Additionally, making such changes without any notification to artists is not good either.

Had a quick discussion with Brecht. The idea is to make it so that warnings will be generated in the interface when requested settings are overwritten and look into adding warnings-as-error type of a toggle for Cycles (with a goal to avoid CPU time waste on the farms, and possibly covering more than just denoiser/device fallback).

Anyway, this is something which is being discussed.

Raimund Klink (Raimund58) added a comment.Jan 28 2022, 3:35 PM

I am looking forward for good solutions/compromises.
You guys are usually quite good in finding those :)

Raimund Klink (Raimund58) changed the task status from Needs Information from Developers to Confirmed.Jan 28 2022, 3:35 PM
Raimund Klink (Raimund58) updated the task description.
Brecht Van Lommel (brecht) changed the subtype of this task from "Report" to "Bug".Feb 4 2022, 7:16 PM