Also it can be fixed by update packaging to 22.0 or higher

from setuptools.

This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install these dependencies. You may be tempted to install setuptools[core], but please avoid doing that as core is an internal implementation detail.

from setuptools.

Oof. This brought all of our CI to a halt, which is the last thing I needed to deal with during a massive migration from gitlab to github.

from setuptools.

This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install [these dependencies]

installing thedependencies in the link works ! please make new build requierment file inseatd of setup tools py and include all dependencies along with setup tools like below: packaging>=24
ordered-set>=3.1.1
more_itertools>=8.8
jaraco.text>=3.7
importlib_resources>=5.10.2
importlib_metadata>=6
tomli>=2.0.1
wheel>=0.43.0

pkg_resources

platformdirs >= 2.6.2
setuptools~=70.0
running this file with apt update and pip install makes the packages run without error

from setuptools.

Per the guidance, please make sure that the setuptools[core] dependencies are installed. In particular, packaging>=24.

from setuptools.

Thanks @jaraco this was helpful. In my project where I was having the issue because on older version of packaging was a dependancy (not compatible with setuptools v71.0.0 or maybe earlier). Updating that dependancy did indeed fix the issue.

It's a bit annoying that the behaviour of setuptools changed which messed up my tests (which gets the latest version of setuptools when setting up) but that is very much my fault and not your teams.

Again, thanks for your comment as it helped my to understand the issue better and what the cause was.

For anyone who has a situation like me where my tests just started failing I'd check to make sure your dependancies and the dependancies of setuptools are compatible.

from setuptools.

Perhaps the CI image I'm using already has packaging installed, but with an older version?

Yes, probably.

Thanks for posting your solution. I'm trying to reason through how we're running into the same issue. We're not specifying packaging anywhere, so I would assume we would get the most current version. And when I install locally, I get the same version of packaging you specify in your PR. Perhaps the CI image I'm using already has packaging installed, but with an older version? I'll investigate and post back if I find something. But it would definitely be a sharp corner if pip install --upgrade setuptools no longer works on ubuntu-latest.

Agreed, it is a bit of a sharp corner, and we made the major release as a signal of this shift in expectations. The issue is that while you're holding it correctly and pip install -U setuptools should theoretically upgrade all the required dependencies, it cannot currently due to the bootstrapping problem. I'm actively working on solving that issue, but in the meantime, one needs to pip install -U setuptools[core] to prepare an environment safely.

You may be tempted to install setuptools[core], but please avoid doing that as core is an internal implementation detail.

I think I may have been too cautious about this advice. We specifically want to avoid packages themselves making this declaration in build-requires. I believe it's safe to use in systems integration tooling.

from setuptools.

Thanks for all the information. This change has broken our CI this morning too. Much as I appreciate the workarounds given here, I would have preferred this behaviour to not have gone mainstream until pypa/packaging-problems#342 is resolved, so I don't have to go back and undo these workarounds in the future.

It may be worth referencing this issue in the quickstart documentation since following the documentation may now not work in many existing environments? https://setuptools.pypa.io/en/latest/userguide/quickstart.html#installation

from setuptools.

That's a really good point. There's a large scope of users affected by this issue but only a small number of system integrators affected by the bootstrapping problem. Perhaps we shouldn't be imposing this extra burden on the larger group of users and should instead push it to integrators (require them, for example, to patch out the declared dependencies if there's no solution to the bootstrap problem).

I agree - updating the installation guide to capture the current state makes sense.

from setuptools.

Can confirm installing 6.1.14 is a temporary solution.
python -m pip install --force-reinstall -v "platformio==6.1.14"

from setuptools.

Hey @jaraco I am using pipenv and running with these versions bellow

pip==24.1.2, setuptools==70.2.0, wheel==0.43.0

But still getting the same error
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'

from setuptools.

Thank you @doctornkz-intelas @jaraco for your inputs and guidance, We are planning to upgrade MWAA to 2.8.1 and I have tested the virtual environment in airflow 2.8.1 the issue is gone as the packaging package came with upgraded version in constraints file.

from setuptools.

Issue is visible when you use packaging<22.0

from setuptools.

Could reproduce the same issue. Fixed by downgrading to 70.x

from setuptools.

This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install these dependencies. You may be tempted to install setuptools[core], but please avoid doing that as core is an internal implementation detail.

Just to confirm, we're seeing this happen in this environment:

python -m pip install --upgrade pip
python -m pip install --upgrade setuptools wheel twine check-wheel-contents

Is that correct? It sounds like you're saying the issue is old dependencies existing in the environment, but we should have the most current packages here.

from setuptools.

Thanks @jaraco this was helpful. In my project where I was having the issue because on older version of packaging was a dependancy (not compatible with setuptools v71.0.0 or maybe earlier). Updating that dependancy did indeed fix the issue.

It's a bit annoying that the behaviour of setuptools changed which messed up my tests (which gets the latest version of setuptools when setting up) but that is very much my fault and not your teams.

Again, thanks for your comment as it helped my to understand the issue better and what the cause was.

For anyone who has a situation like me where my tests just started failing I'd check to make sure your dependancies and the dependancies of setuptools are compatible.

Thanks for posting your solution. I'm trying to reason through how we're running into the same issue. We're not specifying packaging anywhere, so I would assume we would get the most current version. And when I install locally, I get the same version of packaging you specify in your PR. Perhaps the CI image I'm using already has packaging installed, but with an older version? I'll investigate and post back if I find something. But it would definitely be a sharp corner if pip install --upgrade setuptools no longer works on ubuntu-latest.

from setuptools.

@jaraco Thanks for taking the time to respond. This advice is super useful:

You may be tempted to install setuptools[core], but please avoid doing that as core is an internal implementation detail.

I think I may have been too cautious about this advice. We specifically want to avoid packages themselves making this declaration in build-requires. I believe it's safe to use in systems integration tooling.

The issue that we wound up running into is that older versions of our package actually take a runtime dependency on packaging. Upon verifying the package (by installing it), the version of packaging was getting clobbered. This was only causing an issue for the sdist, and not the wheel.

from setuptools.

Another option might be to give the vendored dependencies precedence, but doing so would not reliably solve the issue, as one could python -c "import packaging; import setuptools", which would cause the older packaging to be loaded even thought setuptools would prefer the newer vendored version... and I don't want to go down the road of some intrusive hacks like were employed for distutils (unloading previously-imported packages so they're reloaded).

from setuptools.

We are using AWS MWAA (Airflow) in our stack and it comes with very tight constraints, including packaging ,
I will try to lock setuptools on 70.x.x but maybe there is another way around?

packaging==21.3
    # via
    #   -c constraints.txt
    #   apache-airflow
    #   connexion
    #   huggingface-hub
    #   limits
    #   marshmallow
    #   redshift-connector
    #   sqlalchemy-redshift
    #   tableauserverclient
    ```

from setuptools.

We are using AWS MWAA (Airflow) in our stack and it comes with very tight constraints, including packaging , I will try to lock setuptools on 70.x.x but maybe there is another way around?

Another option is not to install setuptools into that environment. Setuptools is aiming not to be installed in runtime environments and to be used primarily as a build backend. The setuptools distribution does also provide the pkg_resources package, but that too is deprecated. Please consider examining your use of setuptools to see if it can be removed from your dependencies when building these MWAA stack applications. I suspect it's not a direct dependency but a transitive one, in which case please report the issue to the project that's requiring it.

I agree - updating the installation guide to capture the current state makes sense.

This was done in #4507 (quickstart), so I'm closing this issue as there's not much more setuptools can do.

from setuptools.

hi @jaraco , In my case the setuptools. is getting installed as a dependency to snowflake-connector-python==2.9.0 and it is pulling latest version setuptools 72.1.0 which i guess causing the issue.I am afraid force install of specific version of setuptools which may cause issues. This is in MWAA and while creating a virtual environment it is giving the error. Any inputs/guidance is greatly appreciated.

**Executing cmd: /usr/bin/python3.10 -m virtualenv /tmp/cosmos-venvxjhcuezk --python=python3****

**File "/usr/local/airflow/.local/lib/python3.10/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
 INFO -     canonicalize_version(version, strip_trailing_zero=False),
 INFO - TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'**

**subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '-m', 'virtualenv', '/tmp/cosmos-venvxjhcuezk', '--python=python3']' returned non-zero exit status 1.**
 

from setuptools.

I sympathize. It seems that MWAA in particular has very tight constraints (see #4483 (comment)) on packaging.

I'm not familiar enough with that environment to provide an adequate workaround.

@doctornkz-intelas Did you find a workaround? If not, either of you should feel free to file a new ticket describing the specific problem with MWAA and (ideally) providing some repro instructions or describing the interfaces that MWAA gives you to affect the environment. Even better might be to file an issue with MWAA and loop me in there, where I can support the issue in the context of that product. My suspicion is that the long-term solution is to ensure that packaging>=24 is present in the environment, but there are lots of other possible avenues we can pursue (pinning setuptools, building in isolated environments, etc).

from setuptools.

Hugs, @vemulagopal. Yes, we found the way, @jaraco. But it might not be applicable to others.
Introduction: MWAA has very aggressive limits. See details. It is because MWAA is preparing runtime for DAGs and also for itself. You can't upgrade a random library without the risk of breaking MWAA and ruining your day. The best approach for the community to render dependencies with pip-tools compile to get the setup as strict as possible, eg:
pip-compile --constraint=constraints.txt --output-file=requirements.txt requirements.in . After compiling the tool is masking pip and setuptools as not-safe dependencies:

...
# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools

1. Our first very dirty approach was to get risk and add setuptools at the end of the file after rendering. (At that moment I created the comment). It works but the solution is ugly.
2. After discussion with devs we found another way - we are installing setuptools ~70.x.x on the clean CI/CD runner and during rendering dependencies inside venv we are forcefully using system libs. Like that:
   python3 -m venv my_project --system-site-packages. But again it's very tricky and not perfect, but at least setuptools is placed outside of requirements.txt.

If someone knows a more sophisticated approach, please, share - it will help many Airflow users.

from setuptools.