Optimizing collection observes with Oplog about meteor-peerdb HOT 5 CLOSED

Why do you think this would optimize things? So which part would you like to optimize with this? Meteor already uses oplog internally.

from meteor-peerdb.

I read a while back that listening direct to Oblog was more memory efficient then Meteor's observes. I have no idea if that is still, or ever was, true. I tend not to get too close db side of meteor so I am not sure. Was just passing info along just incase you found it useful.

from meteor-peerdb.

Yes. It is very useful. So the question here is a trade-off.

What Meteor (and PeerDB uses that) is currently doing is keeping cache of documents and as oplog changes come in, they update that cache and call observe callbacks. So the question is, what happens if there are millions of documents and we are doing .find({}) observe on whole collection. Does Meteor cache all documents? Or does it have a limit and then if oplog change comes for a document which is not in the cache it fetches it and patches it and sends to observe callbacks?

The other option would be to not have any cache at all, use only oplog, and when we get notification, we use findOne to get a document and use it. So this is probably memory wise the best we can get, but then we will be calling findOne many times. Which is not necessary bad because whole PeerDB assumption is that there are not many writes (and many many more reads).

from meteor-peerdb.

Mind posting that on https://hub.kadira.io/? I think the only way we will know is if we test both cases at scale and that is exactly what that hub is all about.

from meteor-peerdb.

Posting what?

from meteor-peerdb.