Assumptions on Mac OS X break the installation via MacPorts about rawpy HOT 10 CLOSED

Ok, sounds reasonable what you want to do. If you can suggest any particular things to make the setup script more compatible, in this case to MacPorts, then we could work on integrating them. Obviously, all other existing workflows shouldn't break. I don't use Macs, so you will have to help me out here :)

from rawpy.

It would make sense to first discuss the best possible strategy. Mac per se doesn't mean any major difference to Linux other than the fact that barebone OS doesn't come with an option to do "apt-get install libraw/whatever", but there are at least three package manager around that could provide the necessary dependencies.

I would suggest to introduce a configure option that would work on any OS. The option should force either checking for an existing external library or trying to build one. I can imagine that Linux users might also sometimes want to build libraw from sources, either because their system library is too old or maybe it doesn't exist at all or because they don't have the necessary permissions to install it. So the option to build libraw is universally useful, not just on Mac and Windows. On the other hand we also need the opposite: a way to avoid building an internal library on Mac (perhaps also Windows for the very brave?).

Running pkg-config on Mac could be done exactly the same way as on Linux. I'm not sure how that configure file that's only generated by CMake is being used, but while setting it to True broke my Mac build, it's also a bit weird to unconditionally set it to False on Linux.

So rather than checking for "if we run on Mac or Windows", the code should check for "if we should compile libraw". Of course Windows (or potentially Mac) could have a different default value for whether or not to compile libraw and maybe one could try whether a pkg-config search for libraw finds something, but the option should be the one to decide, not the type of the platform alone.

I can try to come up with some code, but I don't want to invent option names and the general strategy that the author wouldn't be comfortable with.

from rawpy.

Do you know of a Python library which would have similar requirements than rawpy and where we could maybe get some inspiration from? I'm not sure about setup.py command line parameters, since this wouldn't quite work when rawpy is installed as part of a dependency of another package I think. But again, I'm not sure how these things are usually done, seems like Python package management is pretty messy in these regards in general, at least for binary packages. My main focus was to get an out-of-the-box experience working for all systems, which typically means binary wheels for Windows and Mac, and looking for an installed library on Linux. But, +1 to get this working properly also on MacPorts etc.

from rawpy.

From some random google searches:

• http://stackoverflow.com/a/2709889
• http://stackoverflow.com/a/680473

A random example that comes to my mind is a module that can depend on different libraries:

• http://autobahn.ws/python/installation.html#install-from-pypi

pip install autobahn[twisted]
pip install autobahn[asyncio]

but I guess this is a different scenario anyway.

I asked on Python's IRC channel. There's too much traffic to get sufficient attention; the only answer was:

The official way is to not build the C library with distutils, but a flag or environment variable is an option, yes.

followed by my question to name a well-written module that does this properly; the reply:

I don't think well-written modules do this.

I would suggest an option like --with-builtin-libraw or --with-libraw=[builtin|path-to-location] or something along those lines.

As a temporary workaround it would help me a lot (even before a special user-visible option would be implemented) if there was some special variable that I could quickly patch, determining whether or not one should build internal libraw or not. Then your setup.py could still use (pseudocode):

if isMac or isWin:
    build_libraw_from_source = True # I could patch this
...
if build_libraw_from_source:
    external_dir = os.path.abspath('external')
    cmake_build = os.path.join(external_dir, 'LibRaw', 'cmake_build')
    ...
else:
    use_pkg_config()
    # don't hardcode this if possible though
    # libraw_config_found = False
    ...

I just noticed that you are actually also fetching cmake on windows and it looks as if only Visual Studio was supported, not to say hardcoded. (Honestly that sounds like a bit of an overkill to me, even though I understand the pain of developing for Windows.) But that's a different issue, let's not mix it here.

from rawpy.

What you linked about --with-featurename and --without-featurename is a deprecated feature of setuptools. In SQLAlchemy version 1.1 the --without-cextensions flag has been removed, and instead they rely on the DISABLE_SQLALCHEMY_CEXT environment variable now.

I guess we could use environment variables then. So the idea would be to have something like ENABLE_RAWPY_LIBRAW_BUILD and I guess this would then also bundle libraw or statically link to it, in all operating systems. I'm not sure how to do the latter under Linux, since this would amount to Linux binary wheels, which are not really a thing yet.

Probably there would then also be another environment variable ENABLE_RAWPY_BUILD_DOWNLOAD which would enable the automatic download of libraw source and necessary build dependencies, which are different for different OS's.

And then further a RAWPY_LIBRAW_SOURCE_PATH which could be used when ENABLE_RAWPY_BUILD_DOWNLOAD is not set and also pkg-config should/cannot be used, so that the path can be specified manually. I would say that this still requires cmake to be present as well.

To create the Mac and Windows default wheels for PyPI, the ENABLE_RAWPY_LIBRAW_BUILD and ENABLE_RAWPY_BUILD_DOWNLOAD variables would have to be set. For your use case, you probably wouldn't set any variable.

You said that libraw_config_found is set to False unconditionally on Linux This is not actually the case. Have a look at the three lines below that. They check if the libraw_config.h file can be found in any of the include dirs and then libraw_config_found is set to True in that case.

Just a quick note about hard-coding Visual Studio: That's just what the official Python distribution requires when you build extensions, it's a binary compatibility (ABI) thing. I'm not considering alternative Windows CPython installations in cygwin or the like.

Before doing any of that it would still be good to have at least one reference in another package that does a similar thing, just to be more confident that this is the right way.

from rawpy.

I'm going to close this issue as I succeeded in producing manylinux1 wheels (https://pypi.python.org/pypi/rawpy/0.10.0a1). Together with the Windows and macOS wheels this should provide sufficient out-of-the-box experience and currently I don't see the necessity for distributing Python packages via package managers other than PyPI.

from rawpy.

The problem is that the package manage then cannot package any software that depends on rawpy. Those can be GUI apps that users would have a hard time configuring themselves. I have to admit that in the meantime I forgot why I needed rawpy in the first place, but I'm sure I needed it to package another piece of software (apparently I didn't need it badly enough to come up with a patch myself :).

from rawpy.

I see, but I don't really understand how this process would scale. You can't publish every PyPI package in all non-pip package managers just in case some software wants to depend on it. Rather it should be possible to either directly depend on Python PyPI packages, or create some wrapper based on the wheels that are available. Re-building each package and including it somewhere else doesn't sound like a good approach to me.

from rawpy.

Well, of course one cannot package every single piece of OpenSource software in every single distribution, but if it helps a lot in packaging when the number of portability issues is smaller when there's a need for packaging. (Last time I tried rawpy was at version 0.7.0, so I would have to check which issues are still present.)

Just today I tried to install one python package with pip, but it depends on numpy (with dependency on Fortran), wxPython and lots of others and I gave up trying to install it on Windows. On our package manager on Mac it was just a matter of <command-name> install <package> and all the dirty details were hidden by hours and hours of cumulated work done by other packagers for other packages.

from rawpy.

from rawpy.