Issues after 1.64 update about boost-feedstock HOT 13 CLOSED

Python2 boost.python libs seem fine on linux with some basic examples:
0de36e4

But yeah, with 1.64 there are fewer libs: only libboost_python.so for py2, libboost_python3.so for py3 whereas it was maybe compiled twice with 1.63 ; maybe we have to find the python3 component explicitely now in cmake if its a py3 interpreter: FIND_PACKAGE(Boost COMPONENTS python3).
b1c3fcd

from boost-feedstock.

@jschueller Thanks a lot for looking into this!

Regarding the fewer libraries, I have been compiling Boost manually rather regularly throughout the last few months on MinGW and, as far as I remember, I always ended up with 2 (I presume identical) versions of boost python, called boost_python and boost_python3. Maybe they changed something in the boost build scripts with 1.64.0?

Looks like calling FIND_PACKAGE(Boost COMPONENTS python3) would work on conda and probably on some (most?) linux distros as well. There are some linux distributions however (like Gentoo, which I am using) which adopt a different nomenclature for the boost python libraries (in an effort, I believe, to cope with Python2 & 3 being installed at the same time), and looking for COMPONENTS python3 will not work :( (just tried it)

Probably the best way to proceed for us is to alter the CI (and, I presume, the recipes down the line when we update from 1.63 to 1.64) to point manually to the correct boost python library. I'll try it out and let you know how it goes.

from boost-feedstock.

They changed this behavior, now it doesn't build a libboost_python.so anymore when built against python3, only the libboost_python3.so, maybe @grafikrobot could confirm ?

It looks gentoo only links boost against python2.7 (https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/boost-build/boost-build-1.63.0.ebuild), archlinux links python2 and python3 (https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/boost), so FIND_PACKAGE(Boost COMPONENTS python3) also works.

from boost-feedstock.

I think @stefanseefeld mentioned something related to this. Maybe he can advise on how to proceed.

from boost-feedstock.

I think @isuruf did some digging at one point and it turned out that on gentoo they also shipped a modified FindBoost.cmake that made things on Gentoo work out-of-the-box wrt the Python 2 vs 3 situation.

from boost-feedstock.

@bluescarni it works out of the box as they ship a vanilla cmake on archlinux.

from boost-feedstock.

How about making a symlink to libboost_python3.so from libboost_python.so? Conda doesn't allow multiple python versions in the same prefix, so there's no problem there.

from boost-feedstock.

@jschueller correct we now only build the BPL for the Python versions actually available. In the past, if only py3 was available, it would build an incorrect libboost_python.* that was intended to be py2, but was actually built against py3. If you want both py2 and py3 versions to be built you will need to configure py2 and py3 (with specified version numbers for each) in whatever *-config.jam you use to get both.

from boost-feedstock.

@isuruf Doing a libboost_python.so => libboost_python3.so would be incorrect as that would make users that expect to be using py2 incorrectly get the py3 version. I.e. please don't do that. If you are having problems because you only have py3 in the OS from some other build system looking for the py2 version you would need to fix that other build system to correctly build against py3 and BPL-py3.

from boost-feedstock.

To reiterate/expand on what @isuruf said, in conda, we only ever (can) have one Python installed in an environment at a time. Further the boost packages correctly identify Python major minor version built against and require that version at run time. That all aside, linking to the symlink will still resolve to the correct libboost_python3 library. So what would be the issue with symlinking the library?

from boost-feedstock.

@jakirkham The issue with the symlink is that it would just hide the problem of the upstream. And it could give the false impression to someone that it's using py2. But I don't know anything about this conda project.. So all I can do is suggest :-)

from boost-feedstock.

I agree, projects using boost.python would have to be patched anyways to compile in regular linux distros.

from boost-feedstock.

Thanks all for helping with this.

I have fixed our scripts to deal with the new situation regarding the naming of the library.

Regarding the other failure we encountered in the CI after the 1.64 update (namely, the ImportError: dynamic module does not define init function (initcore) error on the import of a Boost.Python extension): it turned out it is caused by our use of -fvisibility=hidden, it seems like in 1.64 something changed regarding the visibility attribute in the the declarations of BPL's functions. @grafikrobot Do you know anything about this, i.e., is this the intended behaviour and we should be compiling our Boost Python modules with some preprocessor directive active (or even disable -fvisibility=hidden altogether), or is this a regression? I have been using -fvisibility=hidden for a few years now with Boost.Python and never ran into this issue.

@jschueller We can close this IR as far as I am concerned, thanks a lot for investigating!

from boost-feedstock.