The current implementations of History interface does not allows the "hub" component to scale: The events are stored in local database, that could not be shared across several instances of Hub.
Moreover, the "FindFor" method scan sequentially the database looking for the next items. This implementation would have significant performance impact on a highly used hub with millions of items.
I think that scaling is requirements for the adoption of the project/protocol, it would allows to:
- handle a very large number of concurrent clients
- handle a high throughput of message (or large messages)
- provides edge hub located close to the client and reduce network latency
To solve both issues I suggest few changes in the implementation and protocol specifications:
The uses of a shared Streaming Database (ie. Kafka, AWS Kinesis, Redis 5 stream, ...) to store events would allow running concurrently several instances of Hub.
As several instances of Hub are running, the hub have to also broadcast messages published by other Hub in the cluster. The streaming database (and the MissedEvent feature) could become the source of trust.
With a schema, this is what I'm thinking
# current
/+------+\
+ +
|\+------+/|
| |
| Database |
+------> <-----+
| +----------+ | (init and re-connect)
| |
+---------+-+ +------+ +-+-----------+
| | | | | |
| Publisher | | chan | | Subscriber |
| +------> <-----+ |
+-----------+ +------+ +-------------+
# proposal
+-----------+ +-------------+
| | /---------\ | |
| Publisher | + + | Subscriber |
| +--->\---------/<---+ |
+-----------+ | | +-------------+
+-----------+ | Streaming | +-------------+
| | | Database | | |
| Publisher +---> <---+ Subscriber |
| | +-----------+ | |
+-----------+ +-------------+
To enjoy the benefits of the streaming database, the "last-event-id" should not be generated by the producer (because most of the time such database does not provides primitives to search items, they provide tools to create a cursor at some point in the database (named sequence number) and then traversing the database from that point).
In my opinion, the hub should return the sequence number generated by the streaming database, and the server should take that sequence as the "last-event-id".
If we want to keep the current spec and let the server generating that "last-event-id", the Hub would have to use KeyValue database to convert the "last-event-id" into database sequence number.