GPU will not complete render, CPU does.

[iglooinnovationsinc]

* note, I hope this is not a duplicate post but I tried posting this previously but it did not post*

A render I was working with failed with GPU and yet the CPU renders the scene as expected. The issues are that the GPU takes an unreasonable amount of time to render compared to the CPU and yet the GPU render result has no materials and is largely black with blue components.

My system: Windows 10 64-bit v2004, 3X 1080Ti with nVidia 456.71, C4D R23.008, C4DtoA 3.2

The CPU render, completed in about 4 minutes:

The GPU "render" (this was left for 1hr until I aborted):

Log file: arnold_2020-10-28.txt

To troubleshoot, I conducted the following:

1. Set the lambert as an override shader. GPU fails to render the scene so I aborted after about 20 minutes.
2. Excluded the vehicle, added 4 spheres with a lambert shader. GPU renders scene in seconds
3. Added the scene shaders to the spheres (adding one to each sphere, 4 materials at a time) to see if there was a specific shader at fault. All shaders rendered within reasonable times.
4. Excluded the spheres, and did a GPU render of each component in isolation. GPU rendered each component in isolation within reasonable times. (see below for a few samples)
5. Resumed the full pick-up truck object and initiated GPU render. This is where the GPU render fails with the complete, for some reason.

What I found within the logs the sampling of textures is taking an insane amount of time. I believe in the log after 16 minutes, 2 textures load and from there long spans of time between texture loading and sampling.

I cannot share the vehicle publicly.

GPU rendered components (with respect to step 4 troubleshooting):

[thiago.ize]

The logs are missing the stats, so it's hard to know what's going on. Is it because you interrupted the render? If not, can you try again with more log verbosity? Otherwise, try rendering a smaller res image to see if that completes in a reasonable amount of time so we can get some stats.

Another option is to provide us with a profile json file (see https://docs.arnoldrenderer.com/pages/viewpage.action?pageId=71565401) which we can then use to see where some of the time is being spent.

Have you tried rendering on just one GPU?

[iglooinnovationsinc]

Thanks for the note Thiago. Hopefully these logs and profiles are better: logs.zip

I did not change anything, but I ran the render again with GPU and the result was 52m17s with Camera AA = 1.

For comparison, I configured the scene with Redshift using the equivalent shader setup and textures. The scenes rendered in 15 seconds using Brute force GI for primary and secondary. Granted, it's not 100% equivalent to Arnold:

[thiago.ize]

Thanks. Looks like the render really is this slow. This is not at all expected. It doesn't seem to directly be texture related, since texture reading was fast and there aren't many textures. Textures could indirectly make it slow because of what they do (adjusting roughness changes where rays are fired, etc.).

Can you try rendering with just one GPU? Assuming it's not one of your four GPUs that is causing problems, would you be willing to share the scene privately with the arnold team for us to debug?

[iglooinnovationsinc]

Thiago, here's the render on a single GPU along with the logs. The render completes in 1h 41m 01s. I am waiting for permission to share the model.

cdesse_singlegpu.zip

[strg6004]

Love how you put GPU render in quotes 🙂

Fits!

[iglooinnovationsinc]

Thiago, I have permission to share directly with the Arnold team for debugging. Please let me know how to get it to you.

[thiago.ize]

Great. You can email support@arnoldrenderer.com, mention this thread, and we can figure out through that how to send it over.

[iglooinnovationsinc]

Thanks. Sent an email with a download link.

[thiago.ize]

We can repro the slowdown. It's hard for us to modify the .ass file, but it looks like this might be due to the curvature shader. If you get rid of all curvature nodes does performance go back to normal?

[thiago.ize · Accepted solution]

After looking into this, I'm now pretty confident this is a bug in the curvature shader. While we haven't yet done a proper fix, testing a naive "fix" with your scene makes it render nice and quick.

Until you get a build of arnold with the fixed curvature, you might be able to get better performance by setting all the curvature shaders to use a spread of 1.

[iglooinnovationsinc]

Thanks for the update. I tested this on my end and I confirm using a curvature spread of 1 on all shaders renders the scene in good time. It is worth noting that altering the spread on just 1 material didn't have a large negative impact but if I edited all shaders using curvature to use a spread value other than 1 the scene is impacted as originally posted.

For example

• Camera AA = 1 : 14s
• Camera AA = 2 : 22s
• Camera AA = 6 : 48s (shown below)

[thiago.ize]

The just released Arnold 6.1.0.1 and associated plugins should have this fixed. Let us know if you still see problems.

In the release notes, this is:
core#10128 [GPU] Curvature shader slows down render