lragji / relief-valve Goto Github PK

Need to add support for capped stream, these could be usefull in the fan-out mode where everyone needs a copy of the message and old messages get rolled over.... which also means their accumulated stream should also clear..

Multiple data accumulators will be needed when we need to bucket the data into different sections basically group them ie: Example timeseries batching with time binning..

The same can be achived if we allow different names for accumulation via the constructor argument, it can be an function which can then return Map<batchName,data>

Publisher should have a provision to specify if a specific batch of messages should be consumed by a specific consumer only(a.k.a message routing)

1. Publisher should address the message to a specific consumer by its name [Default: group name+ consumer name]
2. System should have the same guarantee of being delivered in sequence relative to other messages.
3. Special name "*" can be used if the batch needs not be processed by any specific consumer.

This is required when we need variable batch lengths ie when accumalation is resultant of consumer processing time for a batch..
Ref:https://youtu.be/bozAoSIoQpU

There should be a way by which consumers can be configured with how many maximum batches they can keep in flight(a.k.a not acked)

1. Number can vary from 1 to 100(or x but needs an upper limit)
2. Post reaching the number threshold consumers cannot dequeue any batches until the number reduces or a specified time has reached from last update to the counter as this may indicate unexpected consumer deaths/crash