Comments (7)
Can you help explain what it looks like from end-user code? How does the API gets called and gRCP methods/messages get called/sent under-the-hood?
But also, the batching behaviour needs to be turned on by default when you do tx.execute(q)
. That's what I'm expecting this issue to solve. The user can configure batch sizes optionally, but that should be the secondary behaviour.
What I also didn't get from the graph above, is whether each Res
is a batch of Iter.Res
? Is that so?
The only issue with using large batches for a streaming request is that if the client wants to use the Concept API during the transaction, it must wait until it has received the whole of the last batch.
I think this is okay, given that currently, Grakn's transactions are thread-bound anyways.
from typedb-protocol.
which ensures the next batch is being fetched whilst the previous batch is being consumed
If one tx.execute(q)
call gets broken down to multiple batches, I presume that the above should be the case, right?
from typedb-protocol.
In this proposal, does the streaming response have to become async? because as it stands the server can only do a single response per request, i believe. @haikalpribadi i think the proposal isn't to introduce a "message batch" message that is composed of multiple "iter" requests, but rather to stream a bunch of messages from the server in a row...
@adammitchelldev is that right?
from typedb-protocol.
Can you help explain what it looks like from end-user code?
If by end user, you mean user of the grakn client:
The underlying batching is hidden from the user and should happen automatically when the client uses tx.execute
, the change is only an addition to the protocol. This way, the behaviour of the protocol changes without requiring all clients to immediately update to this new method (so we can start with client-java
and then update other clients more gradually). The client APIs will see no changes (unless we choose to expose batching control settings).
How does the API gets called and gRCP methods/messages get called/sent under-the-hood?
The client sends an Iter.Req
message with a batchSize = x
or getAll = true
. The server then sends back a series of Iter.Res
messages. It either sends y
messages, where y == x
or (y <= x
and the last Iter.Res
has done == true
).
The server implementation iterates the query x
times, sending an Iter.Res
each time, before responding to further client messages.
When the client sends a request, it remembers x
(or getAll = true
) and then the RPCIterator
pulls a gRPC message each time. When x
messages have been pulled, the next iteration triggers a new Iter.Req
request and resets x
. If an Iter.Res.done == true
is pulled, the iterator finishes.
What I also didn't get from the graph above, is whether each Res is a batch of Iter.Res? Is that so?
As describe above, each result of the batch is a separate Iter.Res
. This way the results can be received as a stream immediately, making full use of the advantages of gRPC streaming and also requiring no further modifications to the protocol. It's both the easiest way to implement it and also probably the most efficient (since we don't try and re-implement message batching optimisations that gRPC has already implemented).
which ensures the next batch is being fetched whilst the previous batch is being consumed
If one tx.execute(q) call gets broken down to multiple batches, I presume that the above should be the case, right?
Not quite, since in the simple version, the next Iter.Req
batch request is triggered only after x
messages have been received. This means there is a slight "round-trip" stall after each batch, which adds up if the batches are small and the latency is high. In the case where roundTripLatency == batchProcessingTime
, half of your time is wasted. To counter this, you use a very high batch size (like 1,000), so batch processing time is high.
In the double-buffered version, you send 2 Iter.Req.batchSize = x
messages, and whenever you receive the x
results of the first, you issue a new Iter.Req
but continue to pull the Iter.Res
from the other request you made. In the case where roundTripLatency == batchProcessingTime
, you actually hit 100% efficiency, since the server receives the new Iter.Req
right as it finishes responding to the last batch, and the client receives the results from the next Iter.Req
right as it finishes receiving results from the last one.
The second advantage here is that now your batch size can theoretically be much lower, leading to lower buffer sizes and faster responsiveness when the client wants to do something different with the stream (as it would currently need to receive all responses before making a request, which could be seen as a separately fixable issue). I think this should be done as an incremental feature on top of the basic version outlined in this issue though.
In the adaptive version, the client adjusts the batch size dynamically to get as close to roundTripLatency == batchProcessingTime
as possible, another feature that could be incremental.
from typedb-protocol.
In this proposal, does the streaming response have to become async? because as it stands the server can only do a single response per request, i believe. @haikalpribadi i think the proposal isn't to introduce a "message batch" message that is composed of multiple "iter" requests, but rather to stream a bunch of messages from the server in a row...
@adammitchelldev is that right?
Yes, although I think the current implementation would actually be able to handle this without too much adjustment. The issue would be if additional calls (i.e. to concept API) are made during a tx.stream
, since the client would have to flush the last iteration batch before receiving a response to the call. This also introduces the idea that the query is async to the concept API, which may not be desirable.
from typedb-protocol.
Related: https://groups.google.com/forum/#!topic/grpc-io/xbJeiw0BXic
One of the things this proposal relies on is the gRPC layer batching small messages for us. The above issue verifies that as of 2017 the gRPC layer does batch small messages into the same packet, if they get queued fast enough (based on a timer).
from typedb-protocol.
Implemented by #29
from typedb-protocol.
Related Issues (15)
- Replace 'oneof' with 'optional' HOT 1
- Cluster-specific protocol should not be included in Core's protocol HOT 2
- Rename 'Value' (Answer<Value>) to 'Numeric' in Session.proto HOT 1
- Store UUIDs as bytes, not strings
- NodeJS package dependencies should be declared via Bazel
- User role management
- Clean up unused Rust protobuf compilation tooling
- Split up server and client gRPC bindings for Rust HOT 1
- Merge `proto` packages
- Simplify Concept API by replacing `overridden`, `explicit` etc. methods with parameters HOT 1
- Replace RequestID with a tuple of TransactionID + RequestSeq HOT 1
- Remove Null from oneof's HOT 1
- Split ConceptMethod into ThingMethod and TypeMethod HOT 1
- Refactor nodejs compilation from genrule to full bazel rule HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from typedb-protocol.