question

Florian Senand avatar image
Florian Senand asked ·

Arnold Inconsistant and unpredictable Render Times

Hello guys.

I contact you because I use Arnold on C4D since several years now, and I often note very inconsistant and unpredictable render times that make me waste weeks and weeks.

I managed to make several companies (where I often workas a freelancer) to buy some licenses and new machines, and even to orient the all pipeline toward Arnold to share seamlessly project between C4D and 3DSMax. But the matter still the same, whatever the computer, whatever the software, at a certain point of a film production, we had unsolvable increasing render times.

Everything always began correctly. I build my shaders, compare the quality and the render times while I go forward in the project, always trying to keep reasonable computing duration using rayswitch ... And one day, I wake-up, open my project, and render times explode without any reason. This is a bit cartoonish but not so far from the truth.

For exemple, on this project, I made all textures in Mari and rebuild the sader in Arnold. This is one unique shader, only the framing change. One evening, I was at 15 to 30 min for theses frames. The next day morning, I corrected some textures in Mari, exported and reloaded them in Arnold, and relaunch renders. Result : 18min to 3h52 depending on the framing. If you take a look to the 1 and 2, the body fit the same proportion of the picture, only the angle change. However, one angle take 32min, the other 1h48 ! There in no sense ...

An other Exemple. On this project, I have made a first design with minerals. Achieve acceptable render time has been tricky becaused of displace, complex refraction and Ao that allow a metallic shader to propagate over quartz and stones, to blend them evenly. At the middle of the film production, we had to change this design by another, apparently more simple.Only two shaders with a bump, somme SSS, and a bit refractive for one of the them. Result : Same Render Times (if I use Ray switch)


I can continu with another old scene where changing texture size from 4k to 2k divided the render times by 10. But on the same scene, changing 2k by 1k, re-increase render times. Oposite exemple.

On a scene I had 4K substance textures. I opened them in photoshop to make a 2K version, and instead of decrease, render times increased. The same exr output directly in 2k from substance allowed me to reduce times. Last exemple.

I had a scene with rocks, stairs and a tree. All of these are HD geometry with 4k to 8K textures. In solo mode, each one of these elements take above 10 minutes to compute, which that is incredibly fast regarding of there complexity. When they are all acivated in the render, the scene take several hours to render.

Why a so big difference ?

I'm curious to get yours advice on these phoenomes because I really don't know what to do anymore.

c4dtoamaxtoa
carole-36.jpg (724.4 KiB)
mothership.jpg (1.6 MiB)
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Peter Horvath avatar image
Peter Horvath answered ·

Indeed, there's a bug in the code and auto tx replaces the original texture with TX only the first time when the TX is generated, but not when the file already exist. I'm going to fix this ASAP.

About the color space mismatch, auto tx generates a linear texture. It uses the color space setting of the image shader for the conversion. If the color space is 'auto', then it checks the format, floating point image is expected in linear space, so no transform needed, while int textures are expected in sRGB space. Also 8-bit images are converted to a half floating point format.

So what's your original image? 8-bit sRGB? And what's your color space in the image shader? auto?

Or did you generate the tx files in your test manually via the Tx Manager?

12 comments Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Thanks Peter.

I never use the auto mode (sometime It missmatch or change is mind randomly), I always interpret the color space manually to fix it definitivly.

In this case, I setup my color in sRGB, because in mari I painted it in 8bit sRGB (and the other channels in Linear, so I checked Linear in Arnold).

And I used the color space of the node to generate TX.


0 Likes 0 · ·

OK, that explains it, if you explicitly specify sRGB color space in the image node and then use the TX textures generated by auto tx, then you'll have a wrong darker result, because the TX textures are in linear color space.

0 Likes 0 · ·

Ok. So this is a bug too :-)

0 Likes 0 · ·

No, it's not a bug. If you load the linear TX files and specify them as sRGB textures, then the darker result is expected. You have to set the color space to 'linear' or 'auto' to have a correct result with the TX files.

Auto tx should do that automatically after converting the original textures to TX, update the path and set the color space to linear. That's broken currently. So once it's fixed you should get the correct result with auto tx.

0 Likes 0 · ·

I not sure understanding.

If I set my sRGB 8bit tiff to linear, convert it in Tx and re-import it, that give me the opposite issue : result is to Bright.

0 Likes 0 · ·
Show more comments
Show more comments

Btw, I've got a fix for the broken auto tx issue. If you have an Arnold license and entitled to support, I can send you a custom build to test it. Please contact support@arnoldrenderer.com.

0 Likes 0 · ·

Thanks,

I will contact the purchasing manager of my departement, wich will send you a message with the informations you need.

0 Likes 0 · ·
Ramon Montoya Vozmediano avatar image
Ramon Montoya Vozmediano answered ·

One thing that jumps out at me is that your textures are not in .tx format. They should be tiled and mipped for maximum efficiency. From the 3rd log it looks like 85% of time is spent trying to getting texture handles, that might include time spent mipping your textures.

So next thing I would try is using .tx files, They always help, specially if you have 55 high res unmipped maps. Try it and you'll see!

2 comments Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Thanks Ramon.

I tried it lot of time, but I always work in exr and tiff, and .tx never give me really better efficiency.

On the contrary, I often had gamma issies with .tx in the past.

But I will try, again, in case of :-)

1 Like 1 · ·

gamma issues are because the proper color space was not used. If you come across gamma issues again, create a new arnold answers question and we'll help you out with that.

0 Likes 0 · ·
Florian Senand avatar image
Florian Senand answered ·

Ok.

Now that we know Tx was the issue on the character render, that bring us back to the other render, the one I'm working on actually and wich is very, very long too.

The shader is very simple, and there are only 3 or 4 textures that I converted in Tx.

But has Stephen said, among the log, this is not a texture problem.

Wich is very stange is that we have change the desing of the object during the production.

The old one was much more complicate (complex shaders and geometry/displace ...) but globaly faster to compute.

The new one is simple, but very long to compute.

On some shots, there is no difference, but on some other, renders are really long even with bad quality parameters.

This one for exemple take 2h30 for dark reason :

Here is the Log :

Log-OldDesign.txt

Log-NewDesign-V1.txt

Log-NewDesign-V2.txt


ms-compare.jpg (516.6 KiB)
log-olddesign.txt (25.4 KiB)
2 comments Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

These three renders all have different geometry or settings, so I'm not surprised they give different times. It sounds like this might be more of a how to optimize your scene type question? Would you mind reposting this as a new question and marking one of these other answers as solving your inconsistent render times when rendering the exact same scene (aka: use .tx files to fix your perf issue)? This will make it easier for other users with similar problems to find the answer and also ensure we don't lose track of which of your questions still hasn't been answered.

0 Likes 0 · ·

For me, this is the same question actually.

Ok, the geometry is different, but all is more simple on the new design and however, render times exploded.

But for more clarity, I agree that a new thread will be more convinient.

I'll do this :-)

0 Likes 0 · ·
Florian Senand avatar image
Florian Senand answered ·

The issue come from sRGB.

16 bit sRGB dosn't solve the problem.

16 bit linear does :



tx-bug-srgb-lin.jpg (135.9 KiB)
Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Florian Senand avatar image
Florian Senand answered ·

Tx decrease a bit render times on the face, and divide them by 5 on the belly.

But my model is now a black girl :-)

Yes Thiago, this color texture is 8Bit sRGB, the others are at least 16Bit Linear and dosn't seem to bug.

If I delete the Tx, set my color node to linear, and regenerate TX, after force arnold to use them by unchecking "auto convert"+"accept unmipped and untiled", the bug dosn't appear, so the result of the color is too bright :


tx-result.jpg (295.5 KiB)
Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Heribert Raab avatar image
Heribert Raab answered ·

did ever though about switching to linux ? in cases where ray ender time was different from time to time. after I switch to linux the rendering was faster and consistent.

1 comment Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

No, but windows seems to be one of the problems because sometime, I close a project the evening and the day after, the same project don't give me the same results anymore.

So, maybe linux is a solution but not for my companie, unfortunatly :-)

0 Likes 0 · ·
Florian Senand avatar image
Florian Senand answered ·

You're right Thiago Ize. It effectively dosn't use Tx.

I followed Stephen Blair Advise and set these parameter to force C4Dto to use them :

Now use .Tx and seems to be faster, but the famous bug of gamma I was speaking about earlier in this thread, when using .tx appear.

Arnold apply a kind double sRGB gamma on my color texture and changing the input gamma to linear doesn't change anything because it is locked by the Tx file :

Here is the reference color :

And when "use Tx" is activated, it give me this :

I tried to apply a "color correct" node with 2.2 gama, but it dosn't give exactly the same thing as the original texture.

I restart C4D and my computer, but It doesn't work better.

Delete and recreate the Tx, work only on the first render, but the second one became dark again.


tx-rs-new.jpg (19.4 KiB)
tx-bug-color.jpg (62.7 KiB)
1 comment Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Since there seems to be a bug with using existing .tx files, can you try deleting your old .tx files and then regenerate them after changing the input gamma to linear?

Also, it looks like your source textures are 8-bit, is that correct?

0 Likes 0 · ·
Florian Senand avatar image
Florian Senand answered ·

I tried 8 K to complet the test (that will be the size of the final model), but after 10 hours, Arnold computed nothing.

8K .Tx :

CAROLE-AAFinal-Tx-8K.0000.txt


So, I began to test others render engines and decided to use a CPU one.

And Surprise, the same picture with 8K textures compute in only few minutes !

So, again, textures are not the matter, and render time still very random, like I said in the subject of this thread


Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Florian Senand avatar image
Florian Senand answered ·

Worste, I decrease progressively the size of the textures and the gain of rendering time is not so big and overall depend of the camera view.

2K .Tx :

CAROLE-AAFinal-Tx-2K.0000.txt

CAROLE-AAFinal-Tx-2K.0002.txt

1K .Tx :

CAROLE-AAFinal-Tx-1K.0000.txt

CAROLE-AAFinal-Tx-1K.0002.txt



Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Florian Senand avatar image
Florian Senand answered ·

Well.

I converted all my textures in .Tx and has I claimed it lot of time here, it changes ... Nothing ...

Renders are always so long.

4K Textures (Reference Render) :

CAROLE-AAFinal.0000.txt

CAROLE-AAFinal.0002.txt

4K Textures converted in .Tx :

CAROLE-AAFinal-Tx.0000.txt

CAROLE-AAFinal-Tx.0002.txt



5 comments Share
10 |600 characters needed characters left characters exceeded

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

None of these logs are indicating that .tx files were used or at least that they were fully used. You need to use only .tx files, otherwise the non-tx files will still end up dominating. Here's what I see in carole-aafinal-tx0000.txt:

Top files by bytes read:
1 3984.0 MB (14.5%) 4096x4096x3.u8 F:\CAROLE\TEX\CAROLE-Specular.1001.tif UNTILED UNMIPPED MIP-COUNT[77,4,3,2,1,1,2,52,1,1,1,1,3]

Notice how it's a tif file and worse, it's untiled and unmipped -- that is almost guaranteed to make the renderer slower.

I think we need to figure out why .tx files aren't getting used.

0 Likes 0 · ·

Do you have both Use existing .tx textures and Auto-convert textures to TX enabled? Like this?

In my tests, that's when Arnold ends up using the non-tx textures. There's something wrong with how C4DtoA handles those check boxes, and I've logged a bug for it.

To make my scene use tx after I did that (disable auto-convert, enable use existing, and then re-enable auto-convert), I had to disable Auto-Convert and leave Use existing turned on.

1 Like 1 · ·

Yes I checked th boxes and textures has been generated succesfully each time.

I will try your proposition, " I had to disable Auto-Convert and leave Use existing turned on "

0 Likes 0 · ·
tx-ex.jpg (84.7 KiB)
tx-rs.jpg (26.8 KiB)
Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 5 attachments (including images) can be used with a maximum of 2.0 MiB each and 9.8 MiB total.

Welcome to the Arnold Answers community.

This is the place for Arnold renderer users everywhere to ask and answer rendering questions, and share knowledge about using Arnold, Arnold plugins, workflows and developing tools with Arnold.

If you are a new user to Arnold Answers, please first check out our FAQ and User Guide for more information.

When posting questions, please be sure to select the appropriate Space for your Arnold plugin and include the plugin version you are using.

Please include images, scene and log files whenever possible as this helps the community answer your questions.

Instructions for generating full verbosity log files are available for MtoA, MaxtoA, C4DtoA, HtoA, KtoA, and Kick.

If you are looking for Arnold Documentation and Support please visit the Arnold Support site.

To try Arnold please visit the Arnold Trial page.

Bottom No panel present for this section.