circleci-public / cimg-python Goto Github PK
View Code? Open in Web Editor NEWThe Next-Gen CircleCI Python Docker Convenience Image.
Home Page: https://circleci.com/developer/images/image/cimg/python
License: MIT License
The Next-Gen CircleCI Python Docker Convenience Image.
Home Page: https://circleci.com/developer/images/image/cimg/python
License: MIT License
The old circleci/python
images supported alphas, betas and RCs which was really helpful to get library supported before the final release and, indeed, to find bugs in Python before the final releases were made.
This appears not to have been done in the new cimg
images. Please could we have that back?
Here's some example Issues/PRs where these would be helpful:
Since pipenv
is pre-installed on the old Python images, and (I believe) is still the documented way to handle image caching (https://circleci.com/docs/2.0/language-python/#cache-dependencies), would it make sense to bake it into this image as well? The example documented here also won't work (sudo pip install pipenv
), because pip
isn't in root's path (#19 (comment))
I tried switching from circleci/python
to cimg/python
and noticed that it failed because of it not being installed. While it's a relatively simple step to install it, it would be convenient to have it there.
Hi,
What's the process for getting a python 3.11 image up? Can i do anything to help?
Thanks!
I use pipenv
to manage dependencies for a project. With the new version of cimg/python:3.9.15
(sha256:b74e50b094007f94d474df2707b6ce60b7c438eabfce64561a4305ee4a6a72d7
), pipenv
uses version 3.10.6
of Python (vs. the expected 3.9.15
version).
> docker pull cimg/python:3.9.15
3.9.15: Pulling from cimg/python
Digest: sha256:b74e50b094007f94d474df2707b6ce60b7c438eabfce64561a4305ee4a6a72d7
Status: Image is up to date for cimg/python:3.9.15
docker.io/cimg/python:3.9.15
> docker run -it --rm --entrypoint bash cimg/python:3.9.15
circleci@8a86bf0a8874:~/project$ echo $PATH
/home/circleci/.pyenv/shims:/home/circleci/.pyenv/bin:/home/circleci/.poetry/bin:/home/circleci/bin:/home/circleci/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
circleci@8a86bf0a8874:~/project$ pip install pipenv
Requirement already satisfied: pipenv in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (2022.10.25)
Requirement already satisfied: virtualenv in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from pipenv) (20.16.6)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from pipenv) (0.5.7)
Requirement already satisfied: certifi in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from pipenv) (2022.9.24)
Requirement already satisfied: setuptools>=36.2.1 in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from pipenv) (58.1.0)
Requirement already satisfied: platformdirs<3,>=2.4 in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from virtualenv->pipenv) (2.5.2)
Requirement already satisfied: filelock<4,>=3.4.1 in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from virtualenv->pipenv) (3.8.0)
Requirement already satisfied: distlib<1,>=0.3.6 in /home/circleci/.pyenv/versions/3.9.15/lib/python3.9/site-packages (from virtualenv->pipenv) (0.3.6)
circleci@8a86bf0a8874:~/project$ pipenv --version
pipenv, version 2022.10.25
circleci@8a86bf0a8874:~/project$ pipenv install
Creating a virtualenv for this project...
Pipfile: /home/circleci/project/Pipfile
Using /usr/bin/python3 (3.10.6) to create virtualenv...
⠦ Creating virtual environment...created virtual environment CPython3.10.6.final.0-64 in 2302ms
creator Venv(dest=/home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/circleci/.local/share/virtualenv)
added seed packages: pip==22.3, setuptools==65.5.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q
Creating a Pipfile for this project...
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Updated Pipfile.lock (a36a5392bb1e8bbc06bfaa0761e52593cf2d83b486696bf54667ba8da616c839)!
Installing dependencies from Pipfile.lock (16c839)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
circleci@8a86bf0a8874:~/project$ which python
/home/circleci/.pyenv/shims/python
circleci@8a86bf0a8874:~/project$ python --version
Python 3.9.15
circleci@8a86bf0a8874:~/project$ which python3
/home/circleci/.pyenv/shims/python3
circleci@8a86bf0a8874:~/project$ python3 --version
Python 3.9.15
circleci@8a86bf0a8874:~/project$ /usr/bin/python3 --version
Python 3.10.6
The key from these logs is:
Using /usr/bin/python3 (3.10.6) to create virtualenv...
Which is unexpected.
This did not occur with the cimg/python:3.9.14
image:
> docker pull cimg/python:3.9.14
3.9.14: Pulling from cimg/python
Digest: sha256:980d486c5c91d243893b59a9471ad78c2f720cbadf8e87310f9dc8049cb71b98
Status: Image is up to date for cimg/python:3.9.14
docker.io/cimg/python:3.9.14
> docker run -it --rm --entrypoint bash cimg/python:3.9.14
circleci@8014e15a91ae:~/project$ echo $PATH
/home/circleci/.pyenv/shims:/home/circleci/.pyenv/bin:/home/circleci/.poetry/bin:/home/circleci/bin:/home/circleci/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
circleci@8014e15a91ae:~/project$ pip install pipenv
Requirement already satisfied: pipenv in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (2022.9.24)
Requirement already satisfied: setuptools>=36.2.1 in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from pipenv) (58.1.0)
Requirement already satisfied: certifi in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from pipenv) (2022.9.24)
Requirement already satisfied: virtualenv in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from pipenv) (20.16.5)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from pipenv) (0.5.7)
Requirement already satisfied: platformdirs<3,>=2.4 in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from virtualenv->pipenv) (2.5.2)
Requirement already satisfied: filelock<4,>=3.4.1 in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from virtualenv->pipenv) (3.8.0)
Requirement already satisfied: distlib<1,>=0.3.5 in /home/circleci/.pyenv/versions/3.9.14/lib/python3.9/site-packages (from virtualenv->pipenv) (0.3.6)
[notice] A new release of pip available: 22.2.2 -> 22.3
[notice] To update, run: pip install --upgrade pip
circleci@8014e15a91ae:~/project$ pipenv --version
pipenv, version 2022.9.24
circleci@8014e15a91ae:~/project$ pipenv install
Creating a virtualenv for this project...
Pipfile: /home/circleci/project/Pipfile
Using /home/circleci/.pyenv/versions/3.9.14/bin/python3.9 (3.9.14) to create virtualenv...
⠏ Creating virtual environment...created virtual environment CPython3.9.14.final.0-64 in 1703ms
creator CPython3Posix(dest=/home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/circleci/.local/share/virtualenv)
added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q
Creating a Pipfile for this project...
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Updated Pipfile.lock (16c839)!
Installing dependencies from Pipfile.lock (16c839)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
circleci@8014e15a91ae:~/project$ which python
/home/circleci/.pyenv/shims/python
circleci@8014e15a91ae:~/project$ python --version
Python 3.9.14
circleci@8014e15a91ae:~/project$ which python3
/home/circleci/.pyenv/shims/python3
circleci@8014e15a91ae:~/project$ python3 --version
Python 3.9.14
circleci@8014e15a91ae:~/project$ /usr/bin/python3 --version
Python 3.8.10
It used the correct version for the pipenv
:
Using /home/circleci/.pyenv/versions/3.9.14/bin/python3.9 (3.9.14) to create virtualenv...
But, /usr/bin/python3
is oddly 3.8.10 on this image, as well...
circleci@8014e15a91ae:~/project$ /usr/bin/python3 --version
Python 3.8.10
In my opinion, every possible incantation of running python
in the cimg/python
images should always use the version specified in the tag.
Would you accept a PR for supporting Python 3.11 beta releases?
Just over 2 hours ago, the cimg/python:3.9-node
image was updated to point to cimg/python:3.9.8-node, however, the new version updates the major version of nodejs
from 14
to 16
. This caused the build of several of my projects to break.
I downgraded to cimg/python:3.9.7-node
as a temporary solution, however, I would like to know how to use the latest minor version of python3.9.x and a specific version of nodejs
at the same time.
Python 3.10.4 was released recently.
Python 3.10.5 was released recently.
Hello,
I'm trying to migrate our circleci/python:3.7.12-browsers
image to cimg/python:3.7.12-browsers
. You can find my PR here.
All our selenium tests are failing because geckodriver is not in path, but reading cimg images documentation I'm not sure geckodriver is here at all. You can see our test execution results here.
Is it on purpose? Should we install geckodriver manually on every test run, or rollback to legacy browsers image? Thank you for your help <3
Could you please rebuild the Python images to include Poetry 1.2.0 which has been released on August 31?
There are some incompatibility issues about poetry.lock
files being generated by Poetry 1.2.0 and then installed by the Poetry 1.1.7 version included in the current image, and it would help a lot to have the latest Poetry version available.
I use python-magic in my project https://github.com/digitalfabrik/integreat-cms and since the update from 3.9.14
to 3.9.15
, my builds fail with:
import magic
File "/home/circleci/project/.venv/lib/python3.9/site-packages/magic/__init__.py", line 209, in <module>
libmagic = loader.load_lib()
File "/home/circleci/project/.venv/lib/python3.9/site-packages/magic/loader.py", line 49, in load_lib
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
I'm not sure whether it was just a coincidence that libmagic was installed before and now isn't anymore, but just wanted to let you know that the update had this side effect.
For the moment, I just downgraded the image tag. Please let me know whether this was unintentional and will be added again in future versions of this image, or whether I should install this dependency separately in my CircleCI workflow.
Thanks a lot in advance!
Hello
We got problem installing twine on 3.6 due to out-dated pip version.
Is it possible to always upgrade pip on image build to the latest?
It looks like a cleanup in b63a2ef removed xvfb, which the 3.9.15-browsers image relies on.
Please consider setting CONFIGURE_OPTS="--enable-optimizations"
when Python is installed using pyenv, this enables Profile Guided Optimization resulting in a noticeable effect on Python speed.
BTW, official Python Docker images enabled this optimization some time ago, please check docker-library/python#160.
They decided to set PROFILE_TASK
in addition to enabling optimizations when Python <= 3.7 is compiled because of some issues.
When running pytest coverage reports, we keep getting the following error:
Traceback (most recent call last):
File "/home/circleci/repo/tests/modeling/export/test_export_model.py", line 5, in <module>
from skl2onnx.common.data_types import FloatTensorType
File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/__init__.py", line 15, in <module>
from .convert import convert_sklearn, to_onnx, wrap_as_onnx_mixin # noqa
File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/convert.py", line 5, in <module>
from .proto import get_latest_tested_opset_version
File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/proto/__init__.py", line 8, in <module>
from onnx import onnx_pb as onnx_proto # noqa
File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/__init__.py", line 11, in <module>
from onnx.external_data_helper import load_external_data_for_model, write_external_data_tensors, convert_model_to_external_data
File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/external_data_helper.py", line 14, in <module>
from .onnx_pb import TensorProto, ModelProto
File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/onnx_pb.py", line 8, in <module>
from .onnx_ml_pb2 import * # noqa
File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/onnx_ml_pb2.py", line 33, in <module>
_descriptor.EnumValueDescriptor(
File "/home/circleci/.venv/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
------------------------------------------------------------------------------
Ran 0 tests in 2.25s
Exited with code exit status 2
This is with the image python3.9-browsers
, and also occurs when using python3.9.12-browsers
.
I understand that these new images are Ubuntu-based but I'm wondering if they'll ever support operating system distribution names as variants in tags like the legacy images do, e.g cimg/python:3.8-buster
.
In my case, I would prefer it if the image was Debian-based (Buster). Do I have to resort to using the legacy images in order to accomplish this or is there another way?
Will rebuilding this update pip to >= 21.1, and if so, could you all rebuild it for Python 3.x?
We're having our safety
checks fail on
-> pip, installed 21.0.1, affected <21.1, id 40291
There is a compatibility with this combo that is causing problems with some Python tools. For example, as of the time this issue is opened, pipenv
on our cimg/python:3.11.0
images is partially broken. This is an upstream issue among these projects. As fixes/updated dependencies roll out, we'll make a decision if we need to update anything on our end.
Some resources:
When running pipenv run safety check --full-report
with base image cimg/python:3.7.7
in a CircleCI job, getting security finding below:
#!/bin/bash -eo pipefail
pipenv run safety check --full-report
safety report
checked 39 packages, using free DB (updated once a month)
---
-> pip, installed 20.1.1, affected <21.1, id 40291
Pip 21.1 stops splitting on unicode separators in git references, which could be maliciously used to install a different revision on the repository. See: <https://github.com/pypa/pip/issues/9827>. Additionally, pip 21.1 updates urllib3 to 1.26.4 to fix CVE-2021-28363.
--
Exited with code exit status 255
CircleCI received exit code 255
Able to workaround this by running command pipenv run pip3 install --upgrade pip
, but would like to see if pip
can be upgraded.
I'm using this with pyinstaller to build executables and I have to reinstall relevant python b/c it's not installed with CONFIGURE_OPTS=--enable-shared
. Any chance you'd do this by default?
Reference: pyenv/pyenv#392
Thanks!
I was previously using circleci/python:3.8.10
and found no issues with my build, both on PRs and on main branch builds.
- restore_cache: # ensure this step occurs *before* installing dependencies
key: deps9-{{ .Branch }}-{{ checksum "requirements.txt" }}
- run:
name: install requirements
command: |
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
- save_cache:
key: deps9-{{ .Branch }}-{{ checksum "requirements.txt" }}
paths:
- "venv"
As part of migrating from python 3.8 to python 3.10x on my web application, I updated to cimg/python:3.10.4
and there were no issues with PRs - the circleci went through successfully. However, once merged to main branch, it started failing with
#!/bin/bash -eo pipefail
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
Error: [Errno 2] No such file or directory: '/home/circleci/<project name>/venv/bin/python3'
Exited with code exit status 1
CircleCI received exit code 1
I'm suspecting some issues with caches, but i'm unsure how to resolve.
When i tried to SSH into the PR, i can see python3 in blue cyan, which i'm assuming is a symlink.
However, once merged, python3 is in red...
Any ideas?
Hi,
I am trying to run the below script on the 3.10
version of the executor but it keeps prompting for username yet I have the credentials specified under the .nterc
file in my home directory (It should load the credentials from this file).
#!/bin/bash
echo "machine test.com" > /home/circleci/.netrc
echo "login ${USERNAME}" >> /home/circleci/.netrc
echo "password ${API_TOKEN}" >> /home/circleci/.netrc
pip install <package> --extra-index-url ${DOWNLOAD_URL}
Below is the error log when I run the same script on CircleCI Cloud
Looking in indexes: https://pypi.org/simple, https://test.com/artifactory/api/pypi/pypi-local/simple
User for test: ERROR: Exception:
Traceback (most recent call last):
File "/home/circleci/.pyenv/versions/3.10.5/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/home/circleci/.pyenv/versions/3.10.5/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/home/circleci/.pyenv/versions/3.10.5/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 339, in run
requirement_set = resolver.resolve(
File "/home/circleci/.pyenv/versions/3.10.5/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
....
....
File "/home/circleci/.pyenv/versions/3.10.5/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 200, in ask_input
return input(message)
EOFError
WARNING: You are using pip version 22.0.4; however, version 22.2.1 is available.
You should consider upgrading via the '/home/circleci/.pyenv/versions/3.10.5/bin/python3.10 -m pip install --upgrade pip' command.
Exited with code exit status 2
CircleCI received exit code 2
Strangely enough, when I do docker pull
of the same image into my local machine and run the same script, it runs without issues.
Something about the way that Python is installed is causing Python version detection to fail in the SAM CLI. It works in the legacy Python images.
Here are the debug logs for the SAM CLI:
2021-09-10 01:43:20,142 | Invalid executable for python at /home/circleci/.pyenv/shims/python3.9
Traceback (most recent call last):
File "aws_lambda_builders/workflow.py", line 58, in wrapper
File "aws_lambda_builders/workflows/python_pip/validator.py", line 48, in validate
aws_lambda_builders.exceptions.MisMatchRuntimeError: python executable found in your path does not match runtime.
Expected version: python3.9, Found version: /home/circleci/.pyenv/shims/python3.9.
Possibly related: https://github.com/awslabs/aws-lambda-builders/issues/30
2021-09-10 01:43:20,157 | Invalid executable for python at /home/circleci/.pyenv/shims/python
Traceback (most recent call last):
File "aws_lambda_builders/workflow.py", line 58, in wrapper
File "aws_lambda_builders/workflows/python_pip/validator.py", line 48, in validate
aws_lambda_builders.exceptions.MisMatchRuntimeError: python executable found in your path does not match runtime.
Expected version: python3.9, Found version: /home/circleci/.pyenv/shims/python.
Possibly related: https://github.com/awslabs/aws-lambda-builders/issues/30
Here is a code snippet from the aws-lambda-builders module that contains the Python version validation code:
cmd = self._validate_python_cmd(runtime_path)
p = subprocess.Popen(
cmd, cwd=os.getcwd(), stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=OSUtils().original_environ()
)
p.communicate()
if p.returncode != 0:
raise MisMatchRuntimeError(language=self.language, required_runtime=self.runtime, runtime_path=runtime_path)
def _validate_python_cmd(self, runtime_path):
major, minor = self.runtime.replace(self.language, "").split(".") # self.runtime is "python3.9"
cmd = [
runtime_path,
"-c",
"import sys; "
"assert sys.version_info.major == {major} "
"and sys.version_info.minor == {minor}".format(major=major, minor=minor),
]
return cmd
When running the command /home/circleci/.pyenv/shims/python3.9 -c "import sys; assert sys.version_info.major == 3 and sys.version_info.minor == 9"
myself, it passes correctly. I have also opened an issue with the aws-lambda-builders repository, but this issue could be caused by the image and could be causing other unknown issues.
The pyenv functions like pyenv-version-name cannot be executed because /home/circleci/.pyenv/libexec is missing from the PATH
The Docker images created as part of the update to Python 3.9.13 are broken when using /mnt/ramdisk
as working directory, probably because it's based on a version of cimg/base
which is still affected by CircleCI-Public/cimg-base#170. The error message is the same as in the linked issue.
I've already worked around this issue by pinning to sha256:7ab5753b253c17c2de850abdf020fa1ae23a0e5bd2df8b14f5a54ab57d06f4eb
but I figured I'd leave this here in case it helped someone else. At first glance it appears to have something to do with me caching the virtual environment for my project and maybe the location of the python install changing or something.
If I can provide further details let me know.
While I don't think you need/want to add every possible Python tool to the standard image - I think use of virtualenv in CI pipelines is common enough that it makes sense to include in the standard image.
If you have a project with a .python-version in it that is different than the python version included with the image, you'll get strange and mostly broken behavior.
Example:
pip install -e .
for the projectI hope this isn't a bad place to ask this - we're currently using circleci/python:3.8-browsers
and circleci/python:3.8-node-browsers
. As far as I can see there are currently no equivalent variants of these latest-gen cimg/python
images, so unfortunately we can't migrate to them.
Are there any plans to introduce them eventually?
It appears the python install in the python:3.8.6 release was not properly linked against the OS shared libraries. It is still pointing at libffi6, but the OS has libffi7 installed. This causes all kinds of fun problems with python package installs.
python:3.8 is also broken, because it's a floating tag and was updated to point at the new (broken) python:3.8.6 image.
Error from pip install -e .
ImportError: libffi.so.6: cannot open shared object file: No such file or directory
Installed library
$ dpkg -l | grep libffi
ii libffi-dev:amd64 3.3-4 amd64 Foreign Function Interface library (development files)
ii libffi7:amd64 3.3-4 amd64 Foreign Function Interface library runtime
Very odd. Running a 2.7
build locally or on CIrcle fails, python:3.6 seems to work fine.
$ circleci build -c 2dotoh.yml --job codespell
Docker image digest: sha256:81eaab60f3118a29f49ba42229c9a918d792bce8e75e2b2443e4042db932ac87
====>> Spin up Environment
Build-agent version 1.0.18380-12161d8b (2019-11-04T17:44:02+0000)
Docker Engine Version: 19.03.5
Kernel Version: Linux 368961b4dd9b 4.9.184-linuxkit #1 SMP Tue Jul 2 22:58:16 UTC 2019 x86_64 Linux
Starting container cimg/python:2.7
using image cimg/python@sha256:20752b5f66dd6e0447217385eca897649dbc180b96ddf5f1c3792386c06509f3
Using build environment variables
BASH_ENV=/tmp/.bash_env-localbuild-1576771155
CI=true
CIRCLECI=true
CIRCLE_BRANCH=patch-1
CIRCLE_BUILD_NUM=
CIRCLE_JOB=codespell
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
[email protected]:eddiewebb/trinity.git
CIRCLE_SHA1=83ce3cc41ea295a414d211bbd283905665fdc7d1
CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1576771155
CIRCLE_WORKING_DIRECTORY=~/project
The redacted variables listed above will be masked in run step output
====>> Checkout code
#!/bin/bash -eo pipefail
mkdir -p /root/project && cd /tmp/_circleci_local_build_repo && git ls-files | tar -T - -c | tar -x -C /root/project && cp -a /tmp/_circleci_local_build_repo/.git /root/project
====>> install codespell
#!/bin/bash -eo pipefail
pip install codespell
/bin/bash: pip: command not found
Error: Exited with code 127
Step failed
but manually invoking container passes
Development/trinity on patch-1* : [10:34:31]
$ docker run -it cimg/python:2.7 bash
Unable to find image 'cimg/python:2.7' locally
2.7: Pulling from cimg/python
5667fdb72017: Pull complete
d83811f270d5: Pull complete
ee671aafb583: Pull complete
7fc152dfb3a6: Pull complete
e4ca174ff71e: Pull complete
ec7d07481cdc: Pull complete
38808d14ffdd: Pull complete
0f3f034efc2b: Pull complete
5ab14eaa35bf: Pull complete
173f3617fc64: Pull complete
7f41c24f015d: Pull complete
Digest: sha256:20752b5f66dd6e0447217385eca897649dbc180b96ddf5f1c3792386c06509f3
Status: Downloaded newer image for cimg/python:2.7
root@68a64a2e5cee:~/project# pip install codespell
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting codespell
Downloading https://files.pythonhosted.org/packages/df/6f/764ca059e0eb06b69e1abed2c9a2cabe7dac72b336e2600615b38ea547a3/codespell-1.16.0.tar.gz (134kB)
|################################| 143kB 1.2MB/s
Installing collected packages: codespell
Running setup.py install for codespell ... done
Successfully installed codespell-1.16.0
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Hey! Thank's for uploading the -browsers
variants.
If I'm not missing something chrome/chromedriver seems to be missing from the -browsers
images.
I'd expect them to be there based on how the circleci/python/
-browsers
variants worked.
Has the idea of what -browsers
variants should contain changed or is it simply missing as it's still a beta?
It appears for some pip binaries to be made available in PATH, eval "$(pyenv init -)"
needs to be manually run. This needs to be investigated as that doesn't sound ideal.
these variables were in the legacy images, and we found at least PYTHON_VERSION to be useful:
$ docker run -it circleci/python:3.8-buster
$ env | grep PYTHON
PYTHON_PIP_VERSION=21.2.4
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/3cb8888cc2869620f57d5d2da64da38f516078c7/public/get-pip.py
PYTHON_VERSION=3.8.12
PYTHON_SETUPTOOLS_VERSION=57.5.0
PYTHON_GET_PIP_SHA256=c518250e91a70d7b20cceb15272209a4ded2a0c263ae5776f129e0d9b5674309
Running any poetry
command in cimg/python:3.10*
images currently deployed on DockerHub fails with this error:
$ poetry --version
Traceback (most recent call last):
File "/home/circleci/.poetry/bin/poetry", line 17, in <module>
from poetry.console import main
File "/home/circleci/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
from .application import Application
File "/home/circleci/.poetry/lib/poetry/console/application.py", line 3, in <module>
from cleo import Application as BaseApplication
ModuleNotFoundError: No module named 'cleo'
The maintainers should release a new version of the 3.10 images that includes 8367a3b (when building the 3.10 images locally, poetry
works fine).
/bin/bash: coverage: command not found
Exited with code exit status 127
CircleCI received exit code 127
Build running with environment:
Build-agent version 1.0.36552-0105b568 (2020-07-17T13:33:28+0000)
Docker Engine Version: 18.09.6
Kernel Version: Linux 703d9d52b0d1 4.15.0-1077-aws #81-Ubuntu SMP Wed Jun 24 16:48:15 UTC 2020 x86_64 Linux
Starting container cimg/python:2.7.17
image cache not found on this host, downloading cimg/python:2.7.17
2.7.17: Pulling from cimg/python
4877105a: Already exists
caa0f5b9: Already exists
811b6c42: Already exists
5266dcc6: Already exists
f09b727f: Already exists
4fe0a8ff: Already exists
cfc12ba3: Already exists
8eb48b95: Already exists
2bef4971: Already exists
d10ce16b: Already exists
de658b86: Already exists
a9a640b2: Pulling fs layer
f1ab0ed3: Pulling fs layer
Digest: sha256:c38d0b01700e398e50f503290f6ffe084c53456c2f7e66406f5552547350a40227MB/27MB3MBB
Status: Downloaded newer image for cimg/python:2.7.17
pull stats: download 79.36MiB in 833ms (95.25MiB/s), extract 79.34MiB in 3.363s (23.59MiB/s)
time to create container: 532ms
using image cimg/python@sha256:c38d0b01700e398e50f503290f6ffe084c53456c2f7e66406f5552547350a402
Time to upload agent and config: 1.125866351s
Time to start containers: 1.829939611s
My build failed with:
#!/bin/bash -eo pipefail
pip install -r requirements.txt
pyenv: version `2.7.17' is not installed (set by /home/circleci/project/.python-version)
Exited with code exit status 1
CircleCI received exit code 1
Inspecting in SSH shows the inconsistency:
circleci@f3740505c6e8:~$ pyenv versions
system
* 2.7.18 (set by /home/circleci/.pyenv/version)
I've just tried to switch to the cimg/python
image and am getting errors due, I think, to the presence of other Python versions in the image than the named one. (I'm switching because I've also just started seeing this issue with the circleci/python
images).
My use-case is admitedly little unusual -- I'm trying to test a script which auto-detects a version of Python to use (preferring newer ones), however previously I was able to rely on the images containing only a single version of Python and thus validate that the script (including its detection) worked on each version of Python.
Here's an example of a failing build: https://app.circleci.com/pipelines/github/PeterJCLaw/srcomp-dev/35/workflows/6cb3cd62-a361-4ae5-822d-f2ba3a1e32c3
Unfortunately I don't know exactly when the issue started as the code in question changes very slowly; it was updated this week for a security update in an package unrelated to the script which is having issues.
Is it expected that on the 3.7 image there is also a Python 3.8 installed? (for example)
Currently waiting on pyenv to support.
At the moment the cimg
3.9.X variants of Python don't contain the python3.9
site-packages. As an example, here is the contents of /usr/local/lib
on cimg/python:3.9.6
:
ls -la /usr/local/lib
total 0
drwxr-xr-x 1 root root 23 Jun 29 17:14 .
drwxr-xr-x 1 root root 30 Mar 25 16:58 ..
drwxrwsr-x 4 root staff 48 Jun 29 17:15 python2.7
drwxrwsr-x 3 root staff 27 Apr 2 00:16 python3.8
Here is circleci/python:3.9.6
you'll see python3.9
is present:
ls -la /usr/local/lib
total 18512
drwxr-xr-x 1 root root 23 Aug 17 15:38 .
drwxr-xr-x 1 root root 28 Aug 16 00:00 ..
lrwxrwxrwx 1 root root 19 Aug 17 15:38 libpython3.9.so -> libpython3.9.so.1.0
-rwxr-xr-x 1 root root 18939312 Aug 17 15:38 libpython3.9.so.1.0
-rwxr-xr-x 1 root root 15528 Aug 17 15:38 libpython3.so
drwxr-xr-x 2 root root 96 Aug 17 15:38 pkgconfig
drwxr-xr-x 1 root root 266 Aug 17 15:38 python3.9
To replicate:
version: 2.1
jobs:
build:
docker:
- image: circleci/python:3.9.6
steps:
- run: ls -la /usr/local/lib
test:
docker:
- image: cimg/python:3.9.6
steps:
- run: ls -la /usr/local/lib
workflows:
build-and-test:
jobs:
- build
- test
Poetry 1.2.0 has a bug where poetry self update
fails when virtualenv
is upgraded. This started happening today because virtualenv
just released a new version. It can be worked around by running poetry self update
a second time, but it would be great if cimg/python
could push a new version with an upgraded poetry
.
More info: python-poetry/poetry#6062 (comment)
Running poetry --version
on cimg/python:3.10
gives the following error:
docker run cimg/python:3.10 poetry --version
Unable to find image 'cimg/python:3.10' locally
3.10: Pulling from cimg/python
f3ef4ff62e0d: Pull complete
2c322a662952: Pull complete
fb8baef6d8e7: Pull complete
805b7ba0a256: Pull complete
63833a96b952: Pull complete
5d1ffe5f9ba5: Pull complete
2d375d049a3e: Pull complete
8a05961d45f5: Pull complete
67e190f69e82: Pull complete
5704581d4357: Pull complete
Digest: sha256:6a1cde9bd275eeaabd8eee844eb1063c7412d1fc3ce9e8b902fe707fffe6b067
Status: Downloaded newer image for cimg/python:3.10
Traceback (most recent call last):
File "/home/circleci/.poetry/bin/poetry", line 17, in <module>
from poetry.console import main
File "/home/circleci/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
from .application import Application
File "/home/circleci/.poetry/lib/poetry/console/application.py", line 3, in <module>
from cleo import Application as BaseApplication
ModuleNotFoundError: No module named 'cleo'
I reran the job with SSH.
running pyenv
gives me
/home/circleci/.pyenv/libexec/pyenv-help: line 157: exec: pyenv-commands: not found
Title is pretty self descriptive. It would be nice to have a convenient link at the top of the README, or on the Dockerhub page pointing to this repository.
For some reasons, I have to use the pre-released poetry version (1.2.0a2) for a project.
However when I do that I cannot use this circleci image for my tests since it seems to target a 1.1.x versions which is not forward compatible with 1.2.x
What are my options to be able to use this specific poetry version while using this image? Thanks :)
Migrated from circleci\python:3.6
to cimg\python:3.6
and get this
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.