Infer container definition name instead of requiring 'prefect' about prefect-aws HOT 8 CLOSED

Hi! I'm going to move this to prefect-aws where the ECS implementation lives.

from prefect-aws.

Can you please share an executable example that demonstrates your issue? ECSTask can be used without flow runs, e.g.

ECSTask(command=["prefect", "version"]).run()

Then share the details of the task definitions present on AWS?

from prefect-aws.

@madkinsz

I did a little bit more digging and found the issue. If I add a task definition and change the name from "prefect" to something else the resulting task json will contain two task definitions, same as what I displayed above. If you set the "name:" field to "prefect" everything works fine. You can even specify two different images for both task definitions (see second example).

example 1:

ECSTask(
command=["prefect", "version"],
image = "prefecthq/prefect:2.7.9-python3.11",
task_definition = {"containerDefinitions": [
        {
            "name": "test_util_ecs_job",
            "environment": [{"name": "test, "value": "TEST"}]
        }
    ]}
).run()

example 2:

ECSTask(
command=["prefect", "version"],
image = "prefecthq/prefect:2.7.9-python3.11",
task_definition = {"containerDefinitions": [
        {
            "name": "test_util_ecs_job",
            "image": "prefecthq/prefect:2.7.9-python3.9"
            "environment": [{"name": "test", "value": "TEST"}]
        }
    ]}
).run()

"containerDefinitions": [
{
"name": "test_util_ecs_job,
"image": "prefecthq/prefect:2.7.9-python3.9",
"cpu": 0,
"portMappings": [],
"essential": true,
"environment": [
{
"name": "test",
"value": "TEST"
}
],
"mountPoints": [],
"volumesFrom": []
},
{
"name": "prefect",
"image": "prefecthq/prefect:2.7.9-python3.11",
"cpu": 0,
"portMappings": [],
"essential": true,
"environment": [],
"mountPoints": [],
"volumesFrom": []
}
],

from prefect-aws.

I see. Thanks for the additional examples! We have a ticket at #174 to add documentation about this, but we need to know which container definition to set the command for and it is hard-coded to the one named "prefect". As you've noticed, this means you can run separate containers alongside the one that we are managing! We just need to know the name of the one we are supposed to manage. Does that clear things up?

from prefect-aws.

@madkinsz yea it does clear up things for me.
I am not fan of the current implementation would it be feasible to change the behaviour so that it selects user's inputted name and only default to prefect if no name is added.

from prefect-aws.

@rjhawar that could be a breaking change. Why do you care what the container is named?

from prefect-aws.

@madkinsz My concern was more with the behaviour and how strange and non-intuitive it feels:

For instance this is another case where you can potentially run into a problem:
If you were to create your own task definition in AWS and then use the task ARN for your block. If the container definition name wasn't set to "prefect" then you would encounter the same issue.

from prefect-aws.

Hm. Let me poke around and see if there's a reasonable way we can get the behavior you're looking for.

from prefect-aws.