major refactors about go-ssb HOT 4 OPEN

How about merging the Luigi and Margaret packages into this project? It is easier then to iterate and refactor the code base. We can pull them into single packages again when we are happy with their API.

from go-ssb.

Yes! Should probably internalize all these things to make this clearer.

from go-ssb.

Margaret does too much

Another problem is that margaret (our flume-like thingy) also does object decoding and exposes it through empty interfaces, which makes it hard to re-use buffers from previous reads, if all you want is pipe out a bunch of messages, like for replication.

I'm currently dabbling around a version of margaret that just exposes []byte or io.Readers, leaving the marshaling up to the API consumer.

This together with a muxrpc version that gives the user access to reading and writing muxrpc frames without the luigi piping issues outlined above, should result in a pub code that's producing much less overhead.

Currently it's easily a couple hundred megabytes per connection.

from go-ssb.

I'm experimenting with making copies of luigi.Broadcast with concrete types on the new rooms implementation. Much more readable code..!

from go-ssb.