Comments (5)
zanieb
commented on May 23, 2024
1
@boseoladipo np! fyi you can always install from a git branch i.e. pip install git+https://github.com/PrefectHQ/prefect-aws@fix/ecs-log-stream.
from prefect-aws.
harry-hause
commented on May 23, 2024
Also seeing this issue when setting:
stream_output=True,
configure_cloudwatch_logs=True
in the ECSTask Block.
Currently no other logging configuration. I've checked both versions 0.2.0 and 0.2.1.
from prefect-aws.
boseoladipo
commented on May 23, 2024
I also have this issue.
I'm trying to initialize an ECS task object in my flow, and immediately run the ecs task. I have image set to None and configure_cloudwatch_logs set to False to prevent prefect from registering a new task definition, but even with configure_cloudwatch_logs set to True I get the same error.
from prefect import flow, task, get_run_logger
from prefect_aws.ecs import ECSTask
@task
def ecs_flow_task():
ecs_task = ECSTask(
cluster="prefect-default-cluster",
task_definition_arn="arn:aws:ecs:eu-central-1:*********:task-definition/default-adhoc-task",
vpc_id="vpc-*********",
stream_output=True,
configure_cloudwatch_logs=False,
image=None,
task_customizations=[
{
"op": "add",
"path": "/networkConfiguration/awsvpcConfiguration/subnets",
"value": [
"subnet-*********",
"subnet-*********",
"subnet-*********",
],
},
{
"op": "add",
"path": "/networkConfiguration/awsvpcConfiguration/securityGroups",
"value": ["sg-*********"],
},
],
)
ecs_task.run()
logger = get_run_logger()
logger.info("Triggered ECS Task")
@flow()
def ecs_flow():
ecs_flow_task()
if __name__ == "__main__":
ecs_flow()Here's the configuration of the task definition being passed in the fllow:
{
"taskDefinitionArn": "arn:aws:ecs:eu-central-1:*********:task-definition/default-adhoc-task:5",
"containerDefinitions": [
{
"name": "prefect",
"image": "*********.dkr.ecr.eu-central-1.amazonaws.com/adhoc-task",
"cpu": 64,
"memory": 256,
"memoryReservation": 128,
"portMappings": [],
"essential": true,
"environment": [],
"mountPoints": [],
"volumesFrom": [],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "prefect-default-agent-logs",
"awslogs-region": "eu-central-1",
"awslogs-stream-prefix": "adhoc-task-definition"
}
}
}
],
"family": "default-adhoc-task",
"taskRoleArn": "arn:aws:iam::*********:role/prefect-default-ECSTaskRole",
"executionRoleArn": "arn:aws:iam::*********:role/prefect-default-ECSExecutionTaskRole",
"networkMode": "awsvpc",
"revision": 5,
"volumes": [],
"status": "INACTIVE",
"requiresAttributes": [
{
"name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
},
{
"name": "ecs.capability.execution-role-awslogs"
},
{
"name": "com.amazonaws.ecs.capability.ecr-auth"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
},
{
"name": "com.amazonaws.ecs.capability.task-iam-role"
},
{
"name": "ecs.capability.execution-role-ecr-pull"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
},
{
"name": "ecs.capability.task-eni"
}
],
"placementConstraints": [],
"compatibilities": [
"EC2",
"FARGATE"
],
"requiresCompatibilities": [
"FARGATE"
],
"cpu": "512",
"memory": "1024",
"registeredAt": "2022-12-30T21:25:43.736Z",
"deregisteredAt": "2023-01-02T12:34:13.559Z",
"registeredBy": "arn:aws:sts::*******:assumed-role/terraformExecutionRole/aws-go-sdk-1672435543204924000",
"tags": []
}I deploy the flow using a different ECS task definition, saved as an ECS task block in Prefect Cloud. I can see the logs created by the task run, but prefect seems to be trying to read from a log stream that doesn't exist. As far as I can tell the full log stream for task definition run cannot be manipulated by the user, and matches what AWS specified in the documentation.
Is there a basis for prefect trying to read from this log stream:
stream_name = "/".join(
[
log_config["awslogs-stream-prefix"],
task_definition["family"],
task_arn.rsplit("/")[-1],
]
)
Instead of this:
from prefect-aws.
zanieb
commented on May 23, 2024
Thanks for the additional details! Really helpful :)
Here's a fix: #186 — let me know if it works for you!
from prefect-aws.
boseoladipo
commented on May 23, 2024
Thanks for getting this fixed! I guess I'll be able to test this once the fix is merged.
from prefect-aws.
Related Issues (20)
- push_to_s3 and pull_from_s3 do not support additional AwsCredentials block parameters
- AWS S3 copy and move tasks and `S3Bucket` methods
- Prefect Network Configuration defined in work pool fails to get applied in some scenarios when using with ECS Fargate Task HOT 2
- AwsCredentials import raising PydanticUserError HOT 6
- boto3 client use_ssl wrong default
- put_directory creates a new s3_client for each file uploaded HOT 2
- v0.4.4 results in the prefect task to be given the masked PREFECT_API_KEY
- allow define only aws region name through AwsCredentials HOT 1
- Add a Lambda function block HOT 1
- Accept `None` as an argument to Launch Type for ECSTask with `publish_as_work_pool` HOT 1
- Task definition caching does not work if the task definitions come from separate deployments. HOT 2
- Error in version 0.4.8: TypeError: unhashable type: 'dict' HOT 3
- Credential use examples are in correct in documentation HOT 6
- ECS Works Pools Should Support Specifying Volumes for Flow Runs HOT 1
- `S3Bucket.copy_object`: Only resolve target path with self if `to_bucket` is not defined
- ECSTask block `publish_as_work_pool` does not set `network_configuration` HOT 2
- Improvement to ECS worker setup guide
- ECS worker: Updating family setting to template flow name/deployment
- Change logging prefix to avoid unnecessary task definition registrations
- Add Python 3.12 to test matrix
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect-aws.