Texturesys error MtoA

[JoachimKnedel]

Hi, just got a new workstation with Windows 10 on it (all the others PC's here still run Windows7). Fresh install of Maya 2019 and MtoA 3.1.2 (can't install the very latest as we are in the middle of a production).

While rendering on that new workstation I came accross [texturesys] errors which we never had before. Batch render jobs on that PC randomly stopping on random frames with random [texturesys] error messages.

Example would be [texturesys] Read error at row 384, col 512, tile 31; got 0 bytes, expected 88

File was modified after being opened by OIIO (filename = " ......sourceimages/Vandecal02_maintaining3.tx").

The thing is - it could be any of the texture files. Either in standIns, or texture nodes in any Arnold Standard Surface shader. And, the PC is rendering a few frames fine, and then stops randomly with [texturesys] error messages. It seams it is always a ".tx" file, but it could be this texture node or another.

All other Render PC's here running Windows 7 are rendering fine - the same file, same frame range, no errors.

I switched off "Auto-convert Textures to TX" in Render Settings, I reconnected the original jpg-files instead of the .tx-nodes, but doing this for all StandIns too would be a real pain.

Any idea?

[Stephen.Blair]

Other pcs are rendering with the same file? Sounds like they are updating the tx file while it is being read

[JoachimKnedel]

Hi Stephen,

good thinking but looking at the created and modified dates in Windows explorer it looks like this is not the case as these date back as far as January this year and the most recent tx-file is from the beginning of April. And only the new PC running Windows 10 has got the problem with [texturesys].

[Stephen.Blair]

If this is the actual message:

File was modified after being opened by OIIO (filename = " ......sourceimages/Vandecal02_maintaining3.tx").

then I think some other machine is updating the tx file.

Updating a tx file doesn't change the modified date..

[JoachimKnedel]

The below is a typical error message (I abbreviated the file path up the texture file)

00:05:39 1486MB ERROR | [texturesys] Read error at row 832, col 320, tile 109; got 0 bytes, expected 10138
File was modified after being opened by OIIO (filename = "..../Subtle-Grunge-Texture-10.tx")
00:05:39 1428MB WARNING | render terminating early: received abort signal

If the other machines are updating the tx-files, why would only the Windows10 PC have a problem with it? And is there a way of stopping the other machines updating the tx-files if at all possible?

[Stephen.Blair]

You have to disable the Auto-tx conversion in the scene, so that whenever someone works with the scene, it doesn't trigger a tx conversion.

If it's not another user or process, then I would suspect some sort of network issue.

I'm not an expert on file servers...maybe it's something about caching? Or something else that changed in Windows 10...

[thiago.ize]

As Stephen said, it sounds like the .tx files are being regenerated. We only print that error if the file's timestamp changes during a render. Since you saw the jpg timestamp is old, that likely rules out someone modifying the underlying jpg. Since you don't see a change, I think could mean that while the jpg is not modified, the .tx is regenerated due to being passed different settings, and between the time that someone else's Arnold saved the new .tx file and reset its timestamp to match the jpg, your machine read the new file with its not yet reset timestamp.

So things to look for: Is someone rendering with a different version of Arnold (different versions can cause a .tx file to be regenerated)? Is someone changing the .tx file settings causing the file to be regenerated when they render?

[thiago.ize]

By the way, if you have a tool to look at the .tx file's headers (.tx file is just a tiff or exr), you can see what settings were used to generate it. For instance we can see that the following .tx file was built with OIIO 1.7.16 with the following options:

Software: "OpenImageIO 1.7.16 : /usr/local/opt/openimageio/bin/maketx -v --oiio --filter blackman-harris --wrap periodic alumNoise_v056_NRM.tx -o alumNoise_v056_NRM.tx"

That might help you identify if a different version of Arnold is being used or if some setting is different from what the other machines use.

[JoachimKnedel]

I have just double checked the Arnold versions and they are all the same (MtoA 3.1.2, Arnold Core 5.2.2.0).

The texture files were not changed. Auto-generate TX Textures in Attribute editor is off, Auto-convert Textures to TX in Render Settings is off. Don't have a tool to look at the file's header I'm afraid.

We had our IT support already looking into possible network issues as Stephen suggested but all looks fine as far as they can see. Our server runs Windows Small Business Server 2011 - any known problems with Arnold and that OS?

[Stephen.Blair]

Not that I know of. I would copy the textures locally, or create special test copy on the server, and then do some test renders with those.

[thiago.ize]

How about setting the permissions of the jpg/tx files to be read-only (make sure those permissions are followed by all users/computers on the network). Not only should this hopefully fix your errors, but if something tries to modify them, you'll hopefully get a warning alerting you to what the culprit is.

[JoachimKnedel]

That might have worked. However, I was advised from our IT support to use the "Reset this PC" option in Recovery in Windows10 to wipe out the PC. I was not going to do it as it's a brand new workstation and I would need to re-install all the software on it but I did "Reset this PC".

Rewind - I was going to say it worked (as it rendered a few packets in batch render fine), but while writing this it stopped again. Similar error message, but different texture node.

I will need to try the permission settings as suggested and see what happens.

[brurpo]

I did not understand something, is this machine rendering alone? Or is this on a render farm with other PCs?
It really seems like something is opening the file while another something is trying to modify it.
What I usually do before a batch rendering is this:
Open the tx manager. Arnold > Utilities > TX Manager:

Select the folder in your project that contains all your textures, select Search in subfolder and then create .tx. (If you already rendered a frame that already contains all your textures, and you do not have an image sequence, you usually don't have to do this)
(you can also use the Use Scene Textures, and use the folder option only for image sequence and its a good idea to save your color management properly)

Then on the arnold render options disable Auto-convert and keep use existing tx textures:

This will not force you to re-link your textures. It will simply use the latest .tx version of that texture, If you already rendered a frame with all your textures, or made the batch conversion, you will have a .tx for every texture.

Its always good to do this for two reasons. First, you will save render time, especially if your scene has lots of textures, as the different machines will try to convert all the textures every new frame, second, you will avoid problems if two machines tries to convert the same texture at the same time.

You will not have to do anything to your standins.

[JoachimKnedel]

Sorry for the radio silence.

We tried all of the suggested things but nothing "cured" the texturesys error I'm afraid to say.

It's so frustrating as sometimes the machines batch renders say a pack of 10 frames fine using Smedge 2019 as render manager, but mostly it is giving us the error at some point during a pack and then re-renders the same pack fine.

See attached screenshot which illustrates the 2 tries packages always involved the new machine (Jo-Dell4) with Windows 10 on it.

Our IT support recommended installing Windows7 on the new machine if that makes a difference. After a frustrating couple of weeks muddling through a deadline this sounds very tempting.

[Stephen.Blair]

if you're still getting

File was modified after being opened by OIIO (filename = " ......sourceimages/Vandecal02_maintaining3.tx").

then something is updating the file

why not run Process Monitor (either on the file server, the render node, or both) and watch what processes touch the file?

[JoachimKnedel]

A quick update. We have now installed Maya 2019.1 and Arnold 3.2.1 on all the machines here.

I'm currently running Process Monitor on the server and on the Windows 10 machine which gives us the failed batch renders. After doing some test renders via Smedge I'm afraid to say that the error still persists.

Process Monitor is monitoring the sourceimages folder to look out for any changes in that folder while batch rendering and it did record no changes at all to the texture files. Mind you that the texture files are all set to read only.

I think the only thing left to try is to install Windows 7 on the troublesome machine and see what is going to happen.

[JoachimKnedel]

Hi there,

a quick update. After the job which gave us the texturesys errors was finished we updated more workstations here to Windows 10. While batch rendering a different job through Smedge we discovered more and more dropped frames in render packages. No texturesys errors in that job so far though.

Our IT support had a look at this and came across an article which had to do with how Windows 10 deals with mapped drive policy compared to Windows7. This what our support said:

I have found an article online about Windows 10 and group policy applying differently, which could relate to the error you are seeing. Would you be happy for us to make a change to the mapped drive policy which puts the S: Drive on all your computers?

Currently, the policy is to set to "Replace", but it suggests using "Update" which does not change the way it applies on the surface.

Since this change was applied no more problems with dropped frames. And did not came accross a texturesys error either since then.

So hopefully this might be the fix to our texturesys problems we had before. Cheers.