Comments (8)
方案讨论
- client 会在 openTable 的时候去拉取路由表,所以如果想要 warm up,首先需要让用户提前去 openTable,不然就是第一次调用 rpc 的时候自动拉路由表,这样第一次 rpc 会变慢。
调用过程:
- client.openTable(tableName) =>
- clusterManager.openTable(tableName) =>
- new TableHandler(clusterManager, tableName) =>
- queryCfg => tableHandler.initTableConfiguration(),在这里会拉去路由表,初始化 ReplicaSession
if (s.session == null || !s.session.getAddress().equals(pc.primary)) {
// reset to new primary
s.session = manager_.getReplicaSession(pc.primary);
}
- 这里会调用 getReplicaSession,因为以前没创建过连接,所以就 new ReplicaSession()
public ReplicaSession(rpc_address address, EventLoopGroup rpcGroup, int socketTimeout) {
this.address = address;
this.rpcGroup = rpcGroup;
final ReplicaSession this_ = this;
boot = new Bootstrap();
boot.group(rpcGroup)
.channel(ClusterManager.getSocketChannelClass())
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_KEEPALIVE, true)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, socketTimeout)
.handler(
new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel ch) {
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast("ThriftEncoder", new ThriftFrameEncoder());
pipeline.addLast("ThriftDecoder", new ThriftFrameDecoder(this_));
pipeline.addLast("ClientHandler", new ReplicaSession.DefaultHandler());
}
});
this.firstRecentTimedOutMs = new AtomicLong(0);
}
- 注意这里只是会初始化这个连接的数据结构,不会进行 tcp 3次握手,所以理论上现在这条连接还未完全 “warm up”。第一次发 rpc 会检查是否状态为 connected,如果不是就执行
doConnect
,这里才真正进行连接:
VolatileFields cache = fields;
if (cache.state == ConnState.CONNECTED) {
write(entry, cache);
} else {
boolean needConnect = false;
synchronized (pendingSend) {
cache = fields;
if (cache.state == ConnState.CONNECTED) {
write(entry, cache);
} else {
pendingSend.offer(entry);
if (cache.state == ConnState.DISCONNECTED) {
cache = new VolatileFields();
cache.state = ConnState.CONNECTING;
fields = cache;
needConnect = true;
}
}
}
if (needConnect) {
doConnect();
}
我们可以在 getReplicaSession
的时候就提前进行 doConnect
。从而让后面的 rpc 可以跳过握手的环节。
from pegasus-java-client.
上面只是warmup的一方面,即在openTable的时候建立好与MetaServer的connection。实际上完整的warmup应当包括:
- 建立好与MetaServer的connection。
- 获取table的路由表(这里会检查table是否存在等)。
- 根据路由表中各个partition的primary地址(支持backup request之后还应当包括secondary地址),建立好与相关ReplicaServer的connection。
以上过程,接口设计上有两种方式可以选择:
- 在现在PegasusClientInterface::openTable()接口中增加参数
bool warmup
,用户可以在openTable的时候选择是否进行warmup。 - 在PegasusClientInterface增加warmup()接口,用户可以通过调用该方法来进行warmup。我推荐这种。
在实现上,可以考虑这样:
- 在warmup()实现中,对每个partition,伪造rrdb_get_operator,设置hashKey和sortKey都为"",设置gpid为改partition的gpid,设置超时时间(譬如为1秒),然后通过table.asyncOperate()发送请求,并等待回调。在回调中我们并不care结果,因为我们只关心建立connection这个过程。
- 上面hashKey和gpid不需要保持一致,因为不管server端怎么处理,我们只要把查询发到server端,建立connection的目标就已经达到了。
from pegasus-java-client.
@qinzuoyan 你可能误会我的意思,我的思路是这样:
-
用户需要 warm up,就调用 openTable,这也符合这个接口的语义:打开表,查询表信息,建立好所有的连接,做好准备操作。不需要增加 warm_up 接口。
-
openTable 现在不建立连接,所以我们就在拉完路由表之后,对每个 ReplicaSession 调用 doConnect,全部连接都建立完再返回结果。当然这样 openTable 的耗时会增加,我们可以设置建立全部连接的上限时间在 10ms 以内,超过了就说明有慢节点。
-
你只是建立连接,不需要发 fake rpc(这点可以实验一下)
总结就是对延时敏感的用户,让他们提前调 openTable。延时不敏感的用户随便,反正 openTable 多的那一点建立连接的时间对他们来说也无所谓。
后面会发测试情况。
from pegasus-java-client.
时间更长的预热 是指什么意思?
from pegasus-java-client.
时间更长的预热 是指什么意思?
@qinzuoyan 就是用户在实际 send rpc 之前,可能先发 30s 的 fake rpc,不过这样有点类似 ddos。我试了一下,效果不一定好。
from pegasus-java-client.
我觉得其实不需要YCSB来测试很多条查询的P99延迟,只需要对比测试程序启动后第一条查询的延迟是多少就行了,这样更准确。
from pegasus-java-client.
性能测试
- c4 client to c4 server
- ping latency: time=0.046 ms
1. 不提前 openTable
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(283ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(260ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(268ms)
2. 提前 openTable,不提前建立连接
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(18ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(19ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(17ms)
2. 提前 openTable,建立连接,不发 fake rpc
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(14ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(12ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(11ms)
2. 提前 openTable,建立连接,发 fake rpc
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(14ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(10ms)
➜ ./PegasusCli file://./pegasus.properties dup_test set a a a
OK: latency(12ms)
Conclusion
可以看出提前 query meta 可以极大减少延时,提前建立连接也可以一定程度降低延时。但提前发 fake rpc 没有明显提升。这是符合理论的结果。
from pegasus-java-client.
OK,那基本可以这样:在openTable时提前query-meta、并建立连接,但是不需要发fake rpc。
在openTable的时候,需要进行恰当的错误处理,譬如:
- 如果query-meta失败或者表不存在,则抛出异常;
- 如果和replica-server建立连接失败,可以打印日志,但是不抛出异常。
from pegasus-java-client.
Related Issues (20)
- Publish to the Central Repository? HOT 6
- Double-Checked Locking HOT 2
- Unreleased Resource: Streams
- Client API need be refactored and enhanced
- refactor and simplify the API HOT 4
- support option to enable compression and hide details of compression from users HOT 1
- There is a vulnerability in guava 28.1-jre,upgrade recommended
- feature: backup request enhancement HOT 1
- fix extreme deadlock HOT 1
- bug: batchMultiGet may not fetch all data required by the input parameter
- feature: add a app create interface to the java client
- bug: netty rejected task with RejectedExecutionException
- remove log4j from dependencies. HOT 2
- bug
- bug: unable to reconnect a replica session HOT 1
- support backup request on client side HOT 5
- Unreleased Resource: Streams HOT 1
- Double-Checked Locking
- pegasus client logs need to be improved
- Forbid large-size-value writes to Pegasus
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pegasus-java-client.