This is no longer needed, since 'orchestrator' is the only service now.

It might significantly simplify the broker to use kombu. That would mean allowing kombu to use its own event loop and abandoning Twisted's.

I guess one concern would be that we are using deferreds throughout the petri code and that would result in possibly significant shotgun surgery.

The other major concern that I can think of right now is that I'm not sure how to use publisher confirms with kombu's async interface.

I think the primary place we rely on publisher confirms is to quickly publish thousands of create token messages with relatively minimal delay. We wait until all the publisher confirms are received before we ack the message that caused the create token messages.

After some digging through documentation and code, it seems that it might be possible to use publisher confirms the way we want. This library supposedly provides the same interface as librabbitmq, which seems to be exposed directly by kombu.

I believe this is probably possible and would be beneficial, since celery is already using kombu under the hood, so as soon as we start adding celery workers to petri (for example to send http requests), we might start running into weird issues with competing event loops. It is probably a fair bit of work, however.

This will probably involve adding an option to the devserver script to enable coverage gathering.

The python coverage module offers a simple command for combining multiple coverage output files into one.

The only command that's strictly necessary once the broker sets up queues & bindings is the orchestrator command, so the complex command framework can go away.

Simplify the configuration so that it only includes variables for AMQP & Redis connections, host/port information for callback url construction and some options for logging.

Queue/exchange/binding configuration can be made static in code.

According to the celery documentation, it is bad practice to use celery.current_app to get the celery app. The application should be passed to functions that need it instead of being global.

We should eliminate the use of celery.current_app.

I noticed this while trying to write the the split/join test. The orchestrator process simply rejects the message rather than causing a real test failure.

It should allow setting ttl as an action arg.

This storage should contain basic information like net creation time. It can be extended to manage roles when auth is integrated.

When running tox: Unexpected exit of services: code = (0)

In the honcho logs:

ERROR:ptero_petri.implementation.orchestrator.handler_base:Unexpected exception in handler (PetriNotifyPlaceHandler(queue_name=petri_notify_place))
Traceback (most recent call last):
  File "/Users/iferguso/Projects/genome/ptero-petri/ptero_petri/implementation/orchestrator/handler_base.py", line 19, in __call__
    deferred = self._handle_message(message)
  File "/Users/iferguso/Projects/genome/ptero-petri/ptero_petri/implementation/orchestrator/handlers.py", line 38, in _handle_message
    net = get_object(self.redis, message.net_key)
  File "/Users/iferguso/Projects/genome/ptero-petri/ptero_petri/implementation/rom.py", line 647, in get_object
    raise NotInRedisError("No object found in redis with key (%s)" % key)
NotInRedisError: u'No object found in redis with key (ufvmn4G-ThSOI9rbdLreeA)'
ERROR:ptero_petri.implementation.brokers.amqp_broker:Rejecting message (2) due to error: Traceback (most recent call last):
Failure: ptero_petri.implementation.rom.NotInRedisError: u'No object found in redis with key (ufvmn4G-ThSOI9rbdLreeA)'

We've gone back and forth on this a bit in the past (see flow core repo), but it seems most complimentary with using Procfiles to not need to run an additional command to setup RabbitMQ before running foreman/honcho.