尝试使用k8s部署，节点 IP 地址变更问题 about r-nacos HOT 13 CLOSED

回头我把你的helm chart链接放到集群部署说明文档上。

我再调整一下健康检查提一个pr吧， 把这个chart合过来

from r-nacos.

我这几天尽量给一个statefullset方式的k8s部署要样例出来吧。

这是我写的 helm chart helm chart

from r-nacos.

应该是的，我回去再看看怎么解决。

headless service 就可以， dns我集群的问题造成的，已经解决了。

使用 headless service， 重启什么的简单测试了一下没有问题了

from r-nacos.

在k8s部署集群可能要用headless service ，这样每个pod都有一个确定的dns域名,在配置ip的地方换成pod域名。

from r-nacos.

ip变化的节点大于2后，raft协议集群（配置中心）不可写入，distor协议集群(注册中心)以剩余可达的节点继续提供服务。

后台中节点状态对应的是distor节点状态。

from r-nacos.

对k8s的支持按我的计划是在raft性能优化完成后提供支持的。

目前我自己也没在k8s实践过，可以参考eureka在k8s的部署。

from r-nacos.

尝试使用 headless service， dns解析好像会有问题。 第一次集群注册就会失败。 0号节点能够运行起来，但是1号节点无法加入。

tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known

启动节点

• pod-0 配置

RNACOS_RAFT_NODE_ADDR=rnacos-0.rnacos-headles:9848
RNACOS_RAFT_NODE_ID=1
RNACOS_RAFT_JOIN_ADDR = rnacos-0.rnacos-headless:9848

• pod-1配置

RNACOS_RAFT_NODE_ADDR=rnacos-1.rnacos-headles:9848
RNACOS_RAFT_NODE_ID=2
RNACOS_RAFT_JOIN_ADDR = rnacos-0.rnacos-headless:9848

尝试使用 带 .svc.cluster.local:9848的FQDN域名也是同样的结果

完整日志

[2024-01-10T10:13:44.848356Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }
[2024-01-10T10:13:45.128443Z DEBUG rnacos::raft::store::innerstore] inner store handler GetTargetAddr(3)
[2024-01-10T10:13:45.128792Z DEBUG hyper::client::connect::dns] resolving host="rnacos-2.rnacos-headles"
[2024-01-10T10:13:45.136159Z DEBUG tonic::transport::service::reconnect] reconnect::poll_ready: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" }))
[2024-01-10T10:13:45.136228Z DEBUG tower::buffer::worker] service.ready=true message=processing request
[2024-01-10T10:13:45.136254Z DEBUG tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known
[2024-01-10T10:13:45.136332Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }
[2024-01-10T10:13:45.340169Z DEBUG rnacos::raft::store::innerstore] inner store handler GetTargetAddr(2)
[2024-01-10T10:13:45.340526Z DEBUG hyper::client::connect::dns] resolving host="rnacos-1.rnacos-headles"
[2024-01-10T10:13:45.347083Z DEBUG tonic::transport::service::reconnect] reconnect::poll_ready: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" }))
[2024-01-10T10:13:45.347140Z DEBUG tower::buffer::worker] service.ready=true message=processing request
[2024-01-10T10:13:45.347158Z DEBUG tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known
[2024-01-10T10:13:45.347218Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }

from r-nacos.

我这几天尽量给一个statefullset方式的k8s部署要样例出来吧。

from r-nacos.

尝试使用 headless service， dns解析好像会有问题。 第一次集群注册就会失败。 0号节点能够运行起来，但是1号节点无法加入。

tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known

启动节点

• pod-0 配置

RNACOS_RAFT_NODE_ADDR=rnacos-0.rnacos-headles:9848
RNACOS_RAFT_NODE_ID=1
RNACOS_RAFT_JOIN_ADDR = rnacos-0.rnacos-headless:9848

• pod-1配置

RNACOS_RAFT_NODE_ADDR=rnacos-1.rnacos-headles:9848
RNACOS_RAFT_NODE_ID=2
RNACOS_RAFT_JOIN_ADDR = rnacos-0.rnacos-headless:9848

尝试使用 带 .svc.cluster.local:9848的FQDN域名也是同样的结果

完整日志

[2024-01-10T10:13:44.848356Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }
[2024-01-10T10:13:45.128443Z DEBUG rnacos::raft::store::innerstore] inner store handler GetTargetAddr(3)
[2024-01-10T10:13:45.128792Z DEBUG hyper::client::connect::dns] resolving host="rnacos-2.rnacos-headles"
[2024-01-10T10:13:45.136159Z DEBUG tonic::transport::service::reconnect] reconnect::poll_ready: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" }))
[2024-01-10T10:13:45.136228Z DEBUG tower::buffer::worker] service.ready=true message=processing request
[2024-01-10T10:13:45.136254Z DEBUG tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known
[2024-01-10T10:13:45.136332Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }
[2024-01-10T10:13:45.340169Z DEBUG rnacos::raft::store::innerstore] inner store handler GetTargetAddr(2)
[2024-01-10T10:13:45.340526Z DEBUG hyper::client::connect::dns] resolving host="rnacos-1.rnacos-headles"
[2024-01-10T10:13:45.347083Z DEBUG tonic::transport::service::reconnect] reconnect::poll_ready: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" }))
[2024-01-10T10:13:45.347140Z DEBUG tower::buffer::worker] service.ready=true message=processing request
[2024-01-10T10:13:45.347158Z DEBUG tonic::transport::service::reconnect] error: error trying to connect: dns error: failed to lookup address information: Name or service not known
[2024-01-10T10:13:45.347218Z ERROR async_raft_ext::replication] error sending AppendEntries RPC to target|LineRate| error=status: Unknown, message: "transport error: error trying to connect: dns error: failed to lookup address information: Name or service not known", details: [], metadata: MetadataMap { headers: {} }

看日志像域名解析失败

from r-nacos.

我回去之后再看看这个问题

from r-nacos.

看日志像域名解析失败

是的看日志， 应该是 grpc dns 解析时的问题

from r-nacos.

应该是的，我回去再看看怎么解决。

from r-nacos.

回头我把你的helm chart链接放到集群部署说明文档上。

from r-nacos.