It seems that the new version of auditwheel now returns an error code if it is given an non-platform wheel to repair. This causes build-wheels.sh to exit prematurely when processing wheelhouse/*.whl, thus not repairing all the wheels in the wheelhouse.

For my builds, I've just replaced the auditwheel repair line with something like,

auditwheel repair "$whl" --plat $PLAT -w /io/wheelhouse/ || echo "Not a platform-wheel. Continuing..."

Probably not the best solution, but it seems to work.

As a precursor to building a wheel for my own project I wanted to adapt the example to use openblas.

With a few small changes (installing openblas-devel in the container, etc.) I managed to get wheels. However trying to run the code gives segfaults. I resorted to trying to compile and run the small test program below:

#include <stdio.h>
#include <cblas.h>
int main () {
int i;
double x[] = { 1, 2, 3 };
cblas_dscal(3, 4, x, 1);
for (i = 0; i < 3; ++i)
printf ("%f\n", x[i]);
return 0;
}

However even this segfaults with openblas (fine with atlas). A colleague and I spent quite a while with valgrind and gdb in the manylinux centos container and couldn't find the cause.

Are there minimal examples somewhere of using openblas within a wheel? The numpy etc. builds here obviously do it, but I can't decipher the core bits I need to borrow.

Currently the demo doesn't have any pure python dependencies. But if you have any, then those go into the wheelhouse/ folder as well. If you then simply loop with for whl in 'wheelhouse/*.whl' to call auditwheel it will fail for the pure Python packages with "Cannot add platforms to pure wheel". Also, even for binary wheels, I don't want to process any of my dependencies, just my own package, since this is what I would upload to PyPI later, directly from Travis CI. My feeling is that a bit of logic is missing in the demo.

My request: Can the demo be extended to include a small pure Python dependency?

I think in the common use-case you only want to use this config for tags and not for every commit. Is there a way to do that?

Building the manylinux wheels for coverage.py took 1hour 45minutes. Once I removed all my .tox directories, it took 3 minutes.

The problem is that "pip wheel" copies the entire tree somewhere else first. If we use "python setup.py bdist_wheel", it skips that step. If I do that, then even with .tox directories, building wheels takes only 2 minutes.

Here's an IRC exchange about it:

[13:18:39] nedbat	I have two different ways to build wheels:  "pip wheel . -w wheelhouse" and "python setup.py bdist_wheel".  They both produce the same name wheel, with the same size.  But the first command takes 3min42sec, the second command takes 11sec.
[13:19:11] nedbat	The time for the first one seems to depends on how many files are in the entire tree, including .tox directories... What's the difference between these commands, and is there a reason to prefer the first one?
[13:25:42] dstufft	nedbat: the first command copies the files to a temporary directory, does some gross stuff to ensure you're using setuptools in your setup.py, and then invokes setup.py bdist_wheel
[13:26:13] nedbat	dstufft: any idea why it seems take forever when there are many irrelevant files in the tree?
[13:26:18] Wooble	(doesn't pip wheel also recursively build wheels for all of your deps, and bdist_wheel... not?)
[13:26:31] nedbat	Wooble: the results are the same size (though not the same sha1)
[13:26:33] dstufft	nedbat: presumably because we're copying the entire tree to a temporary directory
[13:26:42] nedbat	dstufft: oh, yikes.
[13:27:50] xafer	(cf https://github.com/pypa/pip/issues/2195)
[13:27:59] dstufft	and yea we also build wheels for all dependencies
[13:28:06] nedbat	dstufft: is there a reason i should use "pip wheel" if the other is working for me?
[13:28:07] xafer	(install/wheel, potato/potato)
[13:28:14] dstufft	but those will be seperate .wl files
[13:28:50] dstufft	nedbat: Not particularly, ``pip wheel`` is typically less used by the maintainer of Foo, and more for consumers of Foo-That-Doesnt-Have-A-Wheel-Already
[13:29:14] nedbat	dstufft: ok, good to know.

I'm trying to build a wheel using the manylinux container on travis, but while my call to pip wheel using the /opt/python/cp27-cp27mu/bin/pip builds a wheel, it's excluding .pxd, .pyx, .h, and most importantly, the .so required by my Cython extension, which I'm including in the main package directory, and specifying in my setup.py using extra_link_args=[$ORIGIN]

When I build the wheel on my local Linux container (ubuntu), using pip 8.1.2, all the files are included, the linker picks up my .so adjacent to the .pyx or .c source, correctly generates the Cython .so, and the package works as expected. Running readelf on my Cython module shows Library rpath: [$ORIGIN].

I'm not sure why this omission is only happening in this Docker image, and I realise that auditwheel repair should fix it, but because I'm currently not able to build the .so required by my Cython extension using the manylinux container, I can't use auditwheel. Is there a way I can work around this?

For reference, the Travis manylinux build output is here, the Travis OSX build output (which correctly includes the dylib, in this case) is here and my setup.py is here.

Although this is probably very straightforward for experienced Travis users, but as a newby on Windows it took me a really long time to figure out that my builds were failing because build-wheels.sh needs execute permissions.
As a Windows user, the only way to do that is from a Linux system..

docker: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"exec: \\\"/io/travis/build-wheels.sh\\\": permission denied\"\n".

Something caused the build to timed out. I think it's a one-off but it doesn't look nice to have a broken build. 😄

Could someone please look into this and fix the build?

README explains how to produce wheels via travis but doesn't say anything on how to download them. Any advice?

The docker images have Python 2.6 by default.... this is however a version that starts to be really old and no longer supported AFAIK. Building modern CPython 2.x projects with 2.6 is likely to fail in many cases.

See my terribly bad way to work around it here: https://github.com/pombreda/thirdparty-manylinux/blob/3c35a8fa5eee12dec76993a90632e22cc6dbbe01/travis/build-wheels.sh#L8

I would like to modernize NumPy and SciPy's wheel building/upload scripts in the spirit of current PyPA best practices. We have a dependence on an external C package, use cython internally for our code, and (in the case of NumPy at least) are used as a submodule when building a compatible-but-different wheel by intel. I thought to start with this repo as a base and extend it, but then discovered the other repos under the PyPA which seem to do much of the same thing. Before I start with NumPy, it would be nice to understand the relationships between all of the repos.

In addition to this repo, there is the
pypa/sampleproject
pypa/integration-test
pypa/pip-test-package
pypya/docker-python

It would be nice to consolidate what can be joined together and to have each of these describe its relationship to the others. I could issue PRs to do this but do not really understand which are still considered "best practices".

As long as I am writing about the pypa repo landscape, it seems these repos should be made read-only if not deprecated
pypa/forklift
pypa/pep470
pypa/ssl
pypa/get-virtualenv
pypa/setuptools-issue1494

After building a few wheels with python setup.py and again with pip wheel as in these demo scripts, I found that executable scripts are "cleverly" manipulated on their first line which is set to things like

#!/opt/python/cp27-cp27m/bin/python

This is indeed documented, but the proposed work-around with --executable is not. The original actually said #!/usr/bin/env python which is more portable than what is put in this line...

It is wrong, anyway. If the first line is replaced, then this should not be done with the building platform's hash-bang, but with the target platform's. When I installed one of the wheels from PyPi on another system, this path was indeed still being used (but Python not found there, of course).

Not sure if this is a problem in your code or in setuptools, because the latter lacks in documentation. I'm effectively stuck publishing my packages due to this, so any advise is welcome.

You can see my setup.py if you want.

In my library I don't support Python 2.6 anymore, and I was wondering what the best way is to skip this version in the build-wheels.sh script, instead of the loop for PYBIN in /opt/python/*/bin.