Porting Arnold to Houdini 18.5 Py3

[jaronwilding95]

So I've looked around, and I've seen that Arnold currently doesn't work with Houdini 18.5 Py3 edition, and thats something I'd love to change. I've seen the answers btw, so this is my post trying to get around it all.

I originally posted this up on the Rebelway discord, but I wanted to make it here so I can get help or advice from others. I'll basically copy and paste what I said there though, so that anyone coming in can understand the steps.

Also, this is just a port to get the renderer working, not a proper port with updated methods and functions from python 2 to 3.

This currently is not a working thing, but I've gotten past a ton of different bugs, so I'm putting it here so that someone may be able to help get around it all. Currently there are two types of python modules we need to discuss. The *.py and the *.pyd files. The *.py are standard python modules, which is what we'll be editing. The *.pyd files are c compiled python dll's, optimised with the C language so that they are fast.

Anyway, there are only two things inside the .py files we need to change, and that is the print and has_key functions. These are python 2 functions that have either been remade or just replaced with better ones. So inside our "Arnold" folder, we have several subfolders, those being:

arnold
config
docs
dso
otls
presets
python2.7libs
redist
scripts
soho
toolbar 

Everything we'll be changing scriptwise will be inside the scripts folder. I'll list out all the files that you'll need to change the print function in. This is pretty easy, just whereever it has print somethingtoprinthere change it to print(somethingtoprinthere) (so wrap it in brackets). These files include:

hipfix.py
python\pythonrc.py
python\htoa\conversion.py
python\htoa\device.py
python\htoa\license.py
python\htoa\material.py
python\htoa\ocio.py
python\htoa\parameter.py
python\htoa\session.py

Keep in mind, if there exists a duplicate name with the extension of *.pyc, delete it. Those are pre-compiled versions of the scripts we just changed, and will be run before the *.py, so all our changes will be ignored. Now, for the has_key function. Everywhere it is, for example, if os.environ.has_key('HTOA'):, change it to if "HTOA" in os.environ. You make the has_key changes to the following files:

python\pythonrc.py
python\htoa\object\rop.py
python\htoa\properties.py
python\htoa\searchpath.py
python\htoa\session.py

So that was the relatively easy part done. If I now boot up a py3 version of h18.5, some of those errors should now be gone.

So now those print and environment errors are now gone on load, lets lay down a light. And we get a nice No module named 'psutil' This was a harder fix really, but the initial steps is to copy the folder Arnold\python2.7libs and rename it to Arnold\python3.7libs However, this won't fix some issues. If we dive inside the python3.7libs folder, there is another folder called "psutils", and inside there we find several *.py files and one *.pyd file. The *.pyd file is the one we need to change. If I run it now, we get past the first "no module found" error, but hit a different error saying no DLL was found. This is because the *.pyd file has been compiled for python 2.7. We need the same version compiled for 3.7. So I did some lovely googling, and grabbing of completely wrong files until I found the one I need. Since I run on a windows machine in 64bit, I needed the wheel (python install library basically) for python 3.7 amd64. After grabbing the latest version and trying it, I had to downgrade to version 5.4.8 of psutils for it to work. But installing it is easy, just grab the wheel and then go pip install wheelfile.whl (I'll post the wheel here too, but for reference I used psutil-5.4.8-cp37-cp37m-win_amd64.whl found here -> https://pypi.upc.edu/simple/psutil/). To grab our util, we need to navigate to where our Python3.7 is installed (our custom python that is, not the one with Houdini), where mine is installed under C:\CustomTools\Python\Python37, and navigate to Lib\site-packages. Depending on how much Python you do, you could have a ton of folders in here, or hardly any.

Inside our site-packages, we see two folders with "psutil" in the name.

psutil
psutil-5.4.8.dist-info

We only want the first, psutil Inside there we have several different files, but we mainly just want the one with the *.pyd extension, that being called _psutil_windows.cp37-win_amd64.pyd. Simply rename it to _psutil_windows.pyd and replace the one inside python3.7libs\utils or even better, remove the entire psutil folder from python3.7libs inside Arnold, and copy and paste the psutil folder found inside our site-packages. Now if we lay down our lights or materals, we get no errors. Great! Lets lay down a render node, aaaand another error. This time talking about how a filter object is not subscriptable. This is coming from the file python\htoa\ocio.py. As far as I can see, in python 2 this returned a list, but in python 3, it returns an iterable (meaning it is spawned upon request, so its better for memory). Lets just turn them into a list.

So this fixes everything now, no node-laying errors or anything. The last hiccup is when I get to hitting "Render". Hick fires off, but it fails instantly. I've attached the image result of the failure.

I know this was a massive long read, so if you did read it, and noticed a bunch of spelling mistakes, this was because I copied verbatim the discord post, and only edited some small things here or there.

Thanks to anyone who replies\helps!