[feature request] Add support for '.' characters in TORCH_EXTENSION_NAME about extension-cpp HOT 8 CLOSED

It seems that I compiled PyTorch before the fix.

In [1]: import torch
In [2]: torch.__version__
Out[2]: '0.4.0a0+200fb22'
In [3]: from torch.utils.cpp_extension import BuildExtension
In [4]: import inspect
In [5]: print(inspect.getsource(BuildExtension._define_torch_extension_name))
    def _define_torch_extension_name(self, extension):
        define = '-DTORCH_EXTENSION_NAME={}'.format(extension.name)
        if isinstance(extension.extra_compile_args, dict):
            for args in extension.extra_compile_args.values():
                args.append(define)
        else:
            extension.extra_compile_args.append(define)

from extension-cpp.

I actually found a workaround for now. If you want to name your module "foo.bar" you can use the top-level module name foo as the extension name and put the submodule names in the module def(), as shown here:

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
    m.def("bar.myfunction", &myfunction_cuda, "MY_FUNCTION");
}

from extension-cpp.

cc: @goldsborough

from extension-cpp.

Hi @nkolot, this is a known issue with PYBIND11_MODULE. I thought this was fixed by @fmassa in pytorch/pytorch#6986, where we fixed/hacked it by only making the last part of the dotted extension the actual extension name. I believe setuptools creates the directory structure to still make the extension importable with the full path. Maybe change TORCH_EXTENSION_NAME to just bar? Could you try that? @fmassa do you know why this doesn't work after your fix?

from extension-cpp.

@nkolot are you sure you installed PyTorch from source? The fix @goldsborough mentioned is not present in pytorch 0.4 (which is what we have for the binaries)
Also, can you check that your installation has the fix that @goldsborough mentioned?

from extension-cpp.

Yes, PyTorch is compiled from source.
I can confirm that the fix @goldsborough is working.
So in my setup.py file I have

ext_modules = [CUDAExtension('foo.bar', ['bar_cuda.cpp', 'bar_cuda_kernel.cu']),]

and in bar_cuda.cpp

PYBIND11_MODULE(bar, m) {
    m.def("myfunc", &my_func, "MY_FUNC");
}

It even works with deeper nesting, i.e. foo.bar.foobar, as long as I put the last part in PYBIND.

from extension-cpp.

Great, thanks for confirming. Think we can close this then.

from extension-cpp.

@nkolot I still didn't quite understand... is pytorch/pytorch#6986 present in your installation? That patch is supposed to be doing what @goldsborough mentioned.

from extension-cpp.