Comments (10)
zanieb
commented on May 25, 2024
1
Separately, you don't need to pass security groups to use ECS in this mode so this moto code is just incorrect.
from prefect-aws.
zanieb
commented on May 25, 2024
Is this after a version bump or what?
from prefect-aws.
ahuang11
commented on May 25, 2024
Here's the differences:
{'apprise-1.0.0',
'apprise-1.1.0',
'boto3-1.24.88',
'boto3-1.24.89',
'boto3-stubs-1.24.88',
'boto3-stubs-1.24.89',
'botocore-1.27.88',
'botocore-1.27.89',
'botocore-stubs-1.27.88',
'botocore-stubs-1.27.89',
'greenlet-1.1.3',
'greenlet-1.1.3.post0',
'moto-4.0.6',
'moto-4.0.7',
'pillow-9.2.0',
'prefect-aws-0+untagged.1.gb42b1ff',
'prefect-aws-0+untagged.1.ge6e5b88',
'responses-0.21.0',
'responses-0.22.0',
'types-toml-0.10.8'}
From:
x = set("Jinja2-3.1.2 Mako-1.2.3 Markdown-3.3.7 MarkupSafe-2.1.1 aiofiles-22.1.0 aiohttp-3.8.3 aiosignal-1.2.0 aiosqlite-0.17.0 alembic-1.8.1 anyio-3.6.1 apprise-1.1.0 asgi-lifespan-1.0.1 async-timeout-4.0.2 asyncpg-0.26.0 attrs-22.1.0 black-22.10.0 boto3-1.24.89 boto3-stubs-1.24.89 botocore-1.27.89 botocore-stubs-1.27.89 cachetools-5.2.0 certifi-2022.9.24 cffi-1.15.1 cfgv-3.3.1 charset-normalizer-2.1.1 click-8.1.3 cloudpickle-2.2.0 colorama-0.4.5 commonmark-0.9.1 coolname-1.1.0 coverage-6.5.0 croniter-1.3.7 cryptography-38.0.1 distlib-0.3.6 docker-6.0.0 fastapi-0.85.0 filelock-3.8.0 flake8-5.0.4 frozenlist-1.3.1 fsspec-2022.8.2 ghp-import-2.1.0 google-auth-2.12.0 greenlet-1.1.3.post0 griffe-0.22.2 h11-0.12.0 httpcore-0.15.0 httpx-0.23.0 identify-2.5.6 idna-3.4 iniconfig-1.1.1 interrogate-1.5.0 isort-5.10.1 jmespath-1.0.1 jsonpatch-1.32 jsonpointer-2.3 kubernetes-24.2.0 mccabe-0.7.0 mergedeep-1.3.4 mkdocs-1.4.0 mkdocs-autorefs-0.4.1 mkdocs-gen-files-0.4.0 mkdocs-material-8.5.6 mkdocs-material-extensions-1.0.3 mkdocstrings-0.19.0 mkdocstrings-python-legacy-0.2.3 moto-4.0.7 multidict-6.0.2 mypy-0.982 mypy-extensions-0.4.3 nodeenv-1.7.0 oauthlib-3.2.1 orjson-3.8.0 packaging-21.3 pathspec-0.10.1 pendulum-2.1.2 pillow-9.2.0 platformdirs-2.5.2 pluggy-1.0.0 pre-commit-2.20.0 prefect-2.5.0 prefect-aws-0+untagged.1.ge6e5b88 py-1.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycodestyle-2.9.1 pycparser-2.21 pydantic-1.10.2 pyflakes-2.5.0 pygments-2.13.0 pymdown-extensions-9.6 pyparsing-3.0.9 pytest-7.1.3 pytest-asyncio-0.19.0 pytest-lazy-fixture-0.6.3 python-dateutil-2.8.2 python-slugify-6.1.2 pytkdocs-0.16.1 pytz-2022.4 pytzdata-2020.1 pyyaml-6.0 pyyaml-env-tag-0.1 readchar-4.0.3 requests-2.28.1 requests-oauthlib-1.3.1 responses-0.22.0 rfc3986-1.5.0 rich-12.6.0 rsa-4.9 s3transfer-0.6.0 setuptools-65.4.1 six-1.16.0 sniffio-1.3.0 sqlalchemy-1.4.41 starlette-0.20.4 tabulate-0.9.0 text-unidecode-1.3 toml-0.10.2 tomli-2.0.1 typer-0.6.1 types-awscrt-0.14.7 types-boto3-1.0.2 types-s3transfer-0.6.0.post4 types-toml-0.10.8 typing-extensions-4.4.0 urllib3-1.26.12 uvicorn-0.18.3 virtualenv-20.16.5 watchdog-2.1.9 websocket-client-1.4.1 werkzeug-2.1.2 xmltodict-0.13.0 yarl-1.8.1".split())
y = set("Jinja2-3.1.2 Mako-1.2.3 Markdown-3.3.7 MarkupSafe-2.1.1 aiofiles-22.1.0 aiohttp-3.8.3 aiosignal-1.2.0 aiosqlite-0.17.0 alembic-1.8.1 anyio-3.6.1 apprise-1.0.0 asgi-lifespan-1.0.1 async-timeout-4.0.2 asyncpg-0.26.0 attrs-22.1.0 black-22.10.0 boto3-1.24.88 boto3-stubs-1.24.88 botocore-1.27.88 botocore-stubs-1.27.88 cachetools-5.2.0 certifi-2022.9.24 cffi-1.15.1 cfgv-3.3.1 charset-normalizer-2.1.1 click-8.1.3 cloudpickle-2.2.0 colorama-0.4.5 commonmark-0.9.1 coolname-1.1.0 coverage-6.5.0 croniter-1.3.7 cryptography-38.0.1 distlib-0.3.6 docker-6.0.0 fastapi-0.85.0 filelock-3.8.0 flake8-5.0.4 frozenlist-1.3.1 fsspec-2022.8.2 ghp-import-2.1.0 google-auth-2.12.0 greenlet-1.1.3 griffe-0.22.2 h11-0.12.0 httpcore-0.15.0 httpx-0.23.0 identify-2.5.6 idna-3.4 iniconfig-1.1.1 interrogate-1.5.0 isort-5.10.1 jmespath-1.0.1 jsonpatch-1.32 jsonpointer-2.3 kubernetes-24.2.0 mccabe-0.7.0 mergedeep-1.3.4 mkdocs-1.4.0 mkdocs-autorefs-0.4.1 mkdocs-gen-files-0.4.0 mkdocs-material-8.5.6 mkdocs-material-extensions-1.0.3 mkdocstrings-0.19.0 mkdocstrings-python-legacy-0.2.3 moto-4.0.6 multidict-6.0.2 mypy-0.982 mypy-extensions-0.4.3 nodeenv-1.7.0 oauthlib-3.2.1 orjson-3.8.0 packaging-21.3 pathspec-0.10.1 pendulum-2.1.2 platformdirs-2.5.2 pluggy-1.0.0 pre-commit-2.20.0 prefect-2.5.0 prefect-aws-0+untagged.1.gb42b1ff py-1.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycodestyle-2.9.1 pycparser-2.21 pydantic-1.10.2 pyflakes-2.5.0 pygments-2.13.0 pymdown-extensions-9.6 pyparsing-3.0.9 pytest-7.1.3 pytest-asyncio-0.19.0 pytest-lazy-fixture-0.6.3 python-dateutil-2.8.2 python-slugify-6.1.2 pytkdocs-0.16.1 pytz-2022.4 pytzdata-2020.1 pyyaml-6.0 pyyaml-env-tag-0.1 readchar-4.0.3 requests-2.28.1 requests-oauthlib-1.3.1 responses-0.21.0 rfc3986-1.5.0 rich-12.6.0 rsa-4.9 s3transfer-0.6.0 setuptools-65.4.1 six-1.16.0 sniffio-1.3.0 sqlalchemy-1.4.41 starlette-0.20.4 tabulate-0.9.0 text-unidecode-1.3 toml-0.10.2 tomli-2.0.1 typer-0.6.1 types-awscrt-0.14.7 types-boto3-1.0.2 types-s3transfer-0.6.0.post4 typing-extensions-4.4.0 urllib3-1.26.12 uvicorn-0.18.3 virtualenv-20.16.5 watchdog-2.1.9 websocket-client-1.4.1 werkzeug-2.1.2 xmltodict-0.13.0 yarl-1.8.1".split())
x ^ y
from prefect-aws.
ahuang11
commented on May 25, 2024
Locally, I tested using moto 4.0.7 and 4.0.6. 4.0.7 breaks the test while 4.0.6 does not. Not sure if we want to pin the version or do group_ids=net_conf.get("securityGroups", [])
from prefect-aws.
desertaxle
commented on May 25, 2024
I'd say go with .get
from prefect-aws.
ahuang11
commented on May 25, 2024
Actually, this isn't our code; it's from moto models. https://github.com/rafcio19/moto/blob/05738df4cfa886e256fe91e0732502e759e52690/moto/ecs/models.py#L326-L359
from prefect-aws.
zanieb
commented on May 25, 2024
Let's pin and open an issue upstream? (unless we are misusing)
from prefect-aws.
ahuang11
commented on May 25, 2024
This was committed 11 days ago: https://github.com/spulec/moto/pull/5520/files
Would you like to open an issue upstream? I don't think I understand this enough to report this.
from prefect-aws.
desertaxle
commented on May 25, 2024
It looks like we don't add any security groups to the awsvpcConfiguration that we generate in _load_vpc_network_config which might be the root of this issue. Should we attempt to infer security groups like we do with subnets?
from prefect-aws.
zanieb
commented on May 25, 2024
Apparently security groups are distinct and we can't infer them from the VPC — I asked the same question in #112 (see also #115) :)
from prefect-aws.
Related Issues (20)
- allow define only aws region name through AwsCredentials HOT 1
- Rate limiting on task registration using ECS Workpools
- Add a Lambda function block HOT 1
- Accept `None` as an argument to Launch Type for ECSTask with `publish_as_work_pool` HOT 1
- Make credentials optional for S3Bucket block HOT 2
- Task definition caching does not work if the task definitions come from separate deployments. HOT 2
- Error in version 0.4.8: TypeError: unhashable type: 'dict' HOT 3
- Feature Request: Support for ExtraArgs on S3Bucket
- Credential use examples are in correct in documentation HOT 6
- Add distinct style to links in docs
- ECS Works Pools Should Support Specifying Volumes for Flow Runs HOT 1
- `S3Bucket.copy_object`: Only resolve target path with self if `to_bucket` is not defined
- ECSTask block `publish_as_work_pool` does not set `network_configuration` HOT 2
- Worker mistakenly marks flow runs as Crashed HOT 1
- Improvement to ECS worker setup guide
- ECS worker: Updating family setting to template flow name/deployment
- External luanch type not supported
- Change logging prefix to avoid unnecessary task definition registrations
- Add Python 3.12 to test matrix
- ECS task definition ephemaralStorage overrides do not work
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect-aws.