Hi,
I'm having an issue with AOV passes failing to render on a 28 node Team Render Server controlled Farm.
The project is submitted with display driver present and multiple active AOVs in the Arnold Renderer settings (ref image attached).
When the project is run locally the AOVs along with the Beauty pass as epexted.
But when run on the Team Render Server farm - only produces the main beauty and alpha are returned. It appears to ignore all AOV requests.
Is this a know issue with Team Render? Is there a possible fix/workaround?
We've only recently updated our Render Farm to support Arnold - but this AOV problem has affected all Arnold projects submitted to it.
Our local machines and the render nodes are running:
C4D R21.115
C4DtoA 2.6.1.5 [a79361bf]
Arnold core 5.4.0.2
Any help with the above would be greatly appreciated!
Thanks for your time!
C4DtoA_AOVs.PNGC4DtoA_Output.PNG(I've attached screen grabs of render setting from one of the projects)
Have also tried running the project with the Multipass set to a layered EXR - but this still fails to produce the AOVs while running on the farm.
You might need to add an exr driver for outputting as right now you only have a display driver... see screenshots below, that's my typical output settings and it works over team render
I always bypass C4D's output settings and set the Format to "Arnold drivers only" then set the driver's file path to the path I set in C4D's Mutli-pass settings in the Save section
Thanks so much for the response Eden!
I'll give that a try - much appreciated!
Saving from C4D's output settings should work as well. I made a couple of tests and I don't see any problems. Not sure what's going on.
You're trying to render single frame, right? Does rendering animation any diffrerent?
Can you check the Arnold log? Set log level to Debug in Render Settings > Diagnostics and it should print out the log in the Team Render Client console.
Hi Peter - thanks for the response!
I've run a simple animation test with AOVs saving out through C4D's output settings (not using an Arnold Output Driver) and the AOVs are writing correctly from the farm.
Running that same job as a Single Frame (with all other settings the same) produces the issue of no AOVs being written (just a Beauty file and an alpha file).
I tested Eden's suggestion of an Arnold Output driver (both Tiff and EXR tested) on the original job - but I'm afraid it failed to write the AOVs.
Afraid I'm struggling to get hold of the C4DtoA log - as once submitted to Team Render Server all I am returned is Team Render's log *attached* which looks like it doesn't contain any useful C4DtoA info.
I tried specifying a file path for the Debug log to be written to - but looks like this hasn't worked, might be a write issue our IT side - currently investigating.
job.txt
The arnold log I'm looking for is generated on the clients, not transferred to the server. Should be in the console output of a client.
Can you maybe send me a simple scene? Just to make sure I'm using the same settings.
Thanks for the support Peter!
Please find a link below to the simple scene I'm testing on the farm our end.
As for the Log I can't currently access the clients individually as our Team Render interface is setup using Maxon's web client (image attached)
And currently the only log info available to me are the txt file examples I sent through in the previous message.
I'm talking with our IT department to see if there's any way of accessing the the client log (as you've illustrated through the console) while not running the driving Team Render from a local machine.
There a lots of ERRORs like this:
00:00:15 1707MB ERROR| [c4dtoa] Failed to send data 1037606 to Team Render Server: 1002 00:00:12 1595MB ERROR| [c4dtoa] Failed to send data 1037606 to Team Render Server: 1002
So it's a communication issue between the TeamRender Client and the TeamRender Server
Thanks for information Stephen!
I've run the original Full Res 'single frame' job again with the Arnold Log enabled and it's indeed returning the same Errors.
Understand this might be a 'Team Render' specific issue, but would you have any thoughts on why this might only be happening with 'Single Frame' distributed renders using C4DtoA - but not the case when each node is dealing with a different frame?
I've run the same job using C4D's Physical and Standard renderers on the same farm - and it renders the multipasses as expected.
Animation is managed by C4D, while single frame render requires a custom implementation. Could be a bug in there somewhere.
My feeling is that these errors are not related to the original issue, but some clients trying to get buckets from the server after the render is finished. Can you try to render with less machines, like 5 or 10? Maybe just 1? If the errors disappear that proves my guess.
I'm not able to reproduce the problem on my local network with your scene, AOVs are saved properly. Could you please contact support(at)arnoldrenderer(dot)com and I can send you a build with detailed dev logs to figure out what's wrong when saving the AOV outputs.
Thanks for the suggestion Peter, I'm running a test this afternoon with only 5 nodes active.
I've also dropped an email through to the support@arnold address you reference above - a build with a more detailed log function sounds like it would be really helpful, cheers!