Comments (23)
Also it can be fixed by update packaging to 22.0 or higher
from setuptools.
This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install these dependencies. You may be tempted to install setuptools[core]
, but please avoid doing that as core
is an internal implementation detail.
from setuptools.
Oof. This brought all of our CI to a halt, which is the last thing I needed to deal with during a massive migration from gitlab to github.
from setuptools.
This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install [these dependencies]
installing thedependencies in the link works ! please make new build requierment file inseatd of setup tools py and include all dependencies along with setup tools like below: packaging>=24
ordered-set>=3.1.1
more_itertools>=8.8
jaraco.text>=3.7
importlib_resources>=5.10.2
importlib_metadata>=6
tomli>=2.0.1
wheel>=0.43.0
pkg_resources
platformdirs >= 2.6.2
setuptools~=70.0
running this file with apt update and pip install makes the packages run without error
from setuptools.
Per the guidance, please make sure that the setuptools[core]
dependencies are installed. In particular, packaging>=24
.
from setuptools.
Thanks @jaraco this was helpful. In my project where I was having the issue because on older version of packaging
was a dependancy (not compatible with setuptools v71.0.0 or maybe earlier). Updating that dependancy did indeed fix the issue.
It's a bit annoying that the behaviour of setuptools changed which messed up my tests (which gets the latest version of setuptools when setting up) but that is very much my fault and not your teams.
Again, thanks for your comment as it helped my to understand the issue better and what the cause was.
For anyone who has a situation like me where my tests just started failing I'd check to make sure your dependancies and the dependancies of setuptools are compatible.
from setuptools.
Perhaps the CI image I'm using already has packaging installed, but with an older version?
Yes, probably.
Thanks for posting your solution. I'm trying to reason through how we're running into the same issue. We're not specifying
packaging
anywhere, so I would assume we would get the most current version. And when I install locally, I get the same version ofpackaging
you specify in your PR. Perhaps the CI image I'm using already haspackaging
installed, but with an older version? I'll investigate and post back if I find something. But it would definitely be a sharp corner ifpip install --upgrade setuptools
no longer works onubuntu-latest
.
Agreed, it is a bit of a sharp corner, and we made the major release as a signal of this shift in expectations. The issue is that while you're holding it correctly and pip install -U setuptools
should theoretically upgrade all the required dependencies, it cannot currently due to the bootstrapping problem. I'm actively working on solving that issue, but in the meantime, one needs to pip install -U setuptools[core]
to prepare an environment safely.
You may be tempted to install
setuptools[core]
, but please avoid doing that ascore
is an internal implementation detail.
I think I may have been too cautious about this advice. We specifically want to avoid packages themselves making this declaration in build-requires
. I believe it's safe to use in systems integration tooling.
from setuptools.
Thanks for all the information. This change has broken our CI this morning too. Much as I appreciate the workarounds given here, I would have preferred this behaviour to not have gone mainstream until pypa/packaging-problems#342 is resolved, so I don't have to go back and undo these workarounds in the future.
It may be worth referencing this issue in the quickstart documentation since following the documentation may now not work in many existing environments? https://setuptools.pypa.io/en/latest/userguide/quickstart.html#installation
from setuptools.
That's a really good point. There's a large scope of users affected by this issue but only a small number of system integrators affected by the bootstrapping problem. Perhaps we shouldn't be imposing this extra burden on the larger group of users and should instead push it to integrators (require them, for example, to patch out the declared dependencies if there's no solution to the bootstrap problem).
I agree - updating the installation guide to capture the current state makes sense.
from setuptools.
Can confirm installing 6.1.14 is a temporary solution.
python -m pip install --force-reinstall -v "platformio==6.1.14"
from setuptools.
Hey @jaraco I am using pipenv and running with these versions bellow
pip==24.1.2, setuptools==70.2.0, wheel==0.43.0
But still getting the same error
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
from setuptools.
Thank you @doctornkz-intelas @jaraco for your inputs and guidance, We are planning to upgrade MWAA to 2.8.1 and I have tested the virtual environment in airflow 2.8.1 the issue is gone as the packaging package came with upgraded version in constraints file.
from setuptools.
Issue is visible when you use packaging<22.0
from setuptools.
Could reproduce the same issue. Fixed by downgrading to 70.x
from setuptools.
This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You'll want to either uninstall the older (incompatible) dependencies from the environment or install these dependencies. You may be tempted to install
setuptools[core]
, but please avoid doing that ascore
is an internal implementation detail.
Just to confirm, we're seeing this happen in this environment:
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools wheel twine check-wheel-contents
Is that correct? It sounds like you're saying the issue is old dependencies existing in the environment, but we should have the most current packages here.
from setuptools.
Thanks @jaraco this was helpful. In my project where I was having the issue because on older version of
packaging
was a dependancy (not compatible with setuptools v71.0.0 or maybe earlier). Updating that dependancy did indeed fix the issue.It's a bit annoying that the behaviour of setuptools changed which messed up my tests (which gets the latest version of setuptools when setting up) but that is very much my fault and not your teams.
Again, thanks for your comment as it helped my to understand the issue better and what the cause was.
For anyone who has a situation like me where my tests just started failing I'd check to make sure your dependancies and the dependancies of setuptools are compatible.
Thanks for posting your solution. I'm trying to reason through how we're running into the same issue. We're not specifying packaging
anywhere, so I would assume we would get the most current version. And when I install locally, I get the same version of packaging
you specify in your PR. Perhaps the CI image I'm using already has packaging
installed, but with an older version? I'll investigate and post back if I find something. But it would definitely be a sharp corner if pip install --upgrade setuptools
no longer works on ubuntu-latest
.
from setuptools.
@jaraco Thanks for taking the time to respond. This advice is super useful:
You may be tempted to install
setuptools[core]
, but please avoid doing that ascore
is an internal implementation detail.I think I may have been too cautious about this advice. We specifically want to avoid packages themselves making this declaration in
build-requires
. I believe it's safe to use in systems integration tooling.
The issue that we wound up running into is that older versions of our package actually take a runtime dependency on packaging
. Upon verifying the package (by installing it), the version of packaging
was getting clobbered. This was only causing an issue for the sdist, and not the wheel.
from setuptools.
Another option might be to give the vendored dependencies precedence, but doing so would not reliably solve the issue, as one could python -c "import packaging; import setuptools"
, which would cause the older packaging to be loaded even thought setuptools would prefer the newer vendored version... and I don't want to go down the road of some intrusive hacks like were employed for distutils (unloading previously-imported packages so they're reloaded).
from setuptools.
We are using AWS MWAA (Airflow) in our stack and it comes with very tight constraints, including packaging
,
I will try to lock setuptools on 70.x.x but maybe there is another way around?
packaging==21.3
# via
# -c constraints.txt
# apache-airflow
# connexion
# huggingface-hub
# limits
# marshmallow
# redshift-connector
# sqlalchemy-redshift
# tableauserverclient
```
from setuptools.
We are using AWS MWAA (Airflow) in our stack and it comes with very tight constraints, including
packaging
, I will try to lock setuptools on 70.x.x but maybe there is another way around?
Another option is not to install setuptools into that environment. Setuptools is aiming not to be installed in runtime environments and to be used primarily as a build backend. The setuptools distribution does also provide the pkg_resources
package, but that too is deprecated. Please consider examining your use of setuptools to see if it can be removed from your dependencies when building these MWAA stack applications. I suspect it's not a direct dependency but a transitive one, in which case please report the issue to the project that's requiring it.
I agree - updating the installation guide to capture the current state makes sense.
This was done in #4507 (quickstart), so I'm closing this issue as there's not much more setuptools can do.
from setuptools.
hi @jaraco , In my case the setuptools. is getting installed as a dependency to snowflake-connector-python==2.9.0 and it is pulling latest version setuptools 72.1.0 which i guess causing the issue.I am afraid force install of specific version of setuptools which may cause issues. This is in MWAA and while creating a virtual environment it is giving the error. Any inputs/guidance is greatly appreciated.
**Executing cmd: /usr/bin/python3.10 -m virtualenv /tmp/cosmos-venvxjhcuezk --python=python3****
**File "/usr/local/airflow/.local/lib/python3.10/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
INFO - canonicalize_version(version, strip_trailing_zero=False),
INFO - TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'**
**subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '-m', 'virtualenv', '/tmp/cosmos-venvxjhcuezk', '--python=python3']' returned non-zero exit status 1.**
from setuptools.
I sympathize. It seems that MWAA in particular has very tight constraints (see #4483 (comment)) on packaging.
I'm not familiar enough with that environment to provide an adequate workaround.
@doctornkz-intelas Did you find a workaround? If not, either of you should feel free to file a new ticket describing the specific problem with MWAA and (ideally) providing some repro instructions or describing the interfaces that MWAA gives you to affect the environment. Even better might be to file an issue with MWAA and loop me in there, where I can support the issue in the context of that product. My suspicion is that the long-term solution is to ensure that packaging>=24
is present in the environment, but there are lots of other possible avenues we can pursue (pinning setuptools, building in isolated environments, etc).
from setuptools.
Hugs, @vemulagopal. Yes, we found the way, @jaraco. But it might not be applicable to others.
Introduction: MWAA has very aggressive limits. See details. It is because MWAA is preparing runtime for DAGs and also for itself. You can't upgrade a random library without the risk of breaking MWAA and ruining your day. The best approach for the community to render dependencies with pip-tools compile
to get the setup as strict as possible, eg:
pip-compile --constraint=constraints.txt --output-file=requirements.txt requirements.in
. After compiling the tool is masking pip
and setuptools
as not-safe dependencies:
...
# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools
- Our first very dirty approach was to get risk and add setuptools at the end of the file after rendering. (At that moment I created the comment). It works but the solution is ugly.
- After discussion with devs we found another way - we are installing
setuptools
~70.x.x on the clean CI/CD runner and during rendering dependencies insidevenv
we are forcefully using system libs. Like that:
python3 -m venv my_project --system-site-packages
. But again it's very tricky and not perfect, but at leastsetuptools
is placed outside ofrequirements.txt
.
If someone knows a more sophisticated approach, please, share - it will help many Airflow users.
from setuptools.
Related Issues (20)
- Instead, use pypa/build, pypa/installer or other standards-based tools. HOT 1
- importing `pkg_resources` injects setuptools's vendored dependencies in `sys.path` HOT 3
- ooo
- AttributeError and ModuleNotFoundError after release of 72.x HOT 8
- [BUG] Many packages are no longer installable after test command is removed HOT 127
- [FR] More gradual breakage of setuptools.command.test HOT 17
- [Docs] Missing recommendation on an upper bound version pin for build backends HOT 8
- [FR] Warn when metadata field is defined as dynamic, but not marked in project.dynamic HOT 1
- [Question] AttributeError: 'Extension' object has no attribute '_needs_stub' HOT 3
- Remove use of tag_build and tag_date in Setuptools HOT 6
- The docs build is failing on main.
- Equivalent for distutils.ccompiler.new_compiler needed HOT 2
- [BUG] pip install not working while installing packages defined in setup.py HOT 9
- [BUG] Vulnerabilities blocks installation on setuptools>71.0 HOT 2
- [CI] Inconsistent normalisation of requirement strings on macOS causing trouble in the CI HOT 1
- [BUG] msvc.py never checks `Microsoft.VisualStudio.Component.VC.Tools.arm64` and thus fails to detect MSVC
- Integration tests for `brotli` are failing HOT 2
- [BUG] Installation error importlib_metadata has no attribute 'EntryPoints HOT 3
- [BUG] setuptools version 73.0.0 breaks Pythran HOT 12
- CVE-2024-6345 vulnerability is not fixed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from setuptools.