Handle duplicated messages after connection lost about broadway_rabbitmq HOT 5 CLOSED

Thanks @luos!

@msaraiva I think for now we can leave this issue open and we will work on it once we have more use cases + feedback.

from broadway_rabbitmq.

Hi,

The redelivered flag provides you the information that the message was sent out at least once, but it doesn't provide you if it was actually processed. For example when you view a message on the management interface it is marked as redelivered, so you can't just throw away every redelivered message. If you are having distributed consumers it may be harder to agree on what was actually processed than processing the message again, if that makes sense. You can put a unique id into each message to keep track of the message and record if the processing finished or not.

from broadway_rabbitmq.

Hi @luos!

Thanks! That was very helpful. It looks like handling duplicated messages will add a lot of complexity to Broadway, especially due to the possibility that the message can be redelivered to another distributed consumer. So I believe we should keep this out of Broadway's scope. However, I think we should provide a way to allow the user to implement his own solution when necessary. One idea would be to define a handle_unacknowledged_message callback. @josevalim WDYT?

from broadway_rabbitmq.

Btw, I think this is no longer relevant now that we metadata. With metadata, we can expose the redelivered flag to users and then they can act accordingly, isn't this correct?

from broadway_rabbitmq.

What about adding some option called delivery_method? With values :at_most_once, :at_least_once. In case someone doesn't want to or doesn't know how to use metadata.

edit: Sorry, just noticed this has already been considered. #37

from broadway_rabbitmq.