Entities: cfg: Configuration from yaml Task Manager: exposes task decorator and maintains global list of tasks Task: A task that can be dispatched. The decorated function AsyncInvocation: An instance of calling a task AsyncResult: The future for retrieving a task result

@task decorator enrolls the method into task_manager._ALL_TASKS task.dispatch() -> create Message -> publisher.publish() -> SQS consumer.message_handler() -> create Message from JSON data -> task_manager.find_task_by_name() -> task.call(message) -> calls task function

Questions: How is the connection to SQS managed? Persistent? does Session() return the same connection each call?

Worker:

Pool of N workers One fetcher process per SQS queue we are reading from The fetcher processes enqueue (multiprocessing.Queue) the Message for a worker to get() The worker calls the task in the Message The worker deletes the message from the queue

reply_to: each task has it's own reply_to RequestMessage and ResponseMessage are separate entities Each publisher registers a reply queue per task

• Make sure publishers delete reply_to queues when exiting