Possible to create a Python subclass in libpython-clj? about libpython-clj HOT 3 CLOSED

Hello, I was able to port the MNIST example from pytorch/examples successfully.
I was probably able to avoid the decref issue by making the class a root namespace definition.
The code is here: https://github.com/raymond-w-ko/techniko/blob/mnist/src/techniko/mnist.clj
The repo is here: https://github.com/raymond-w-ko/techniko/tree/mnist
I also added some instructions so hopefully it can serve as a full example on how to use pytorch to train a model in Clojure.

One issue I experienced is that I had to use a stack-resource-context to ensure GC is done at the end of a train or test batch loop, otherwise my GPU would quickly run out of VRAM. If you look at https://github.com/raymond-w-ko/techniko/blob/mnist/src/techniko/mnist.clj#L173 it gets tricky, because pytorch wants you to use a python with on torch.no_grad to internally disable gradient calculation (original example here: https://github.com/pytorch/examples/blob/master/mnist/main.py#L55).

If the test loop is replaced with any of the commented out code blocks, I get a cryptic python error in flight.

I can understand the first example throwing, since you are doing a double GIL lock since py/with does a GIL lock.

I do not understand why the second example throws since I didn't have time to deeply dive into the code.

from libpython-clj.

Sorry for the noise, I just saw the tests, which has the py/create-class fn. This should be enough for me to figure it out.

from libpython-clj.

Let me know how that goes, no one has tried that yet. When we bridge containers into python, we do subclass the appropriate abc classes, however, so it is lightly tested but as with anything in python we don't know how well we are really doing it. I will say that you want to be careful to ensure your objects don't go out of scope else you may find a decref call ruining your day.

from libpython-clj.