Faced the following problem when using grpc-lb-consul:

• gRPC service runs on some HOST:PORT
• then it exits
• another (HTTP/1) service starts on same HOST:PORT (really rare case, but it happened with me)
• balancer continues to query that HOST:PORT (with new HTTP/1 service running) and logs many errors:

error retrieving load score for EXITED_GRPC_SERVICE from HOST:PORT: rpc error: code = 13 desc = transport is closing

It's logged here: balancer/backend.go#L79

New HTTP/1 service receives HTTP/2 PRI method requests (which it cannot process; but even if it could - another error would be returned - like "no such endpoint" or similar).

My suggestion is to close backend on errors (probably, only on some specific errors - like "closing" or similar) and exclude it from a set of backends for service.

Backends are re-discovered (so closed/connected) each balancer.Config.Discovery.Interval (defaults to 5m).

So even if there is no new service, running on that HOST:PORT, grpc-lb-consul may flood the logs with some other errors (like "couldn't connect" or so).

So "self-closing" backends would be nice to have.

Also, probably, health/load checks would be better done with gRPC streams? Not that it's very important, but if another gRPC service "reuses" address, it may be reported to "previous" service client while services are not re-discovered. But this is rare case, so, probably, not worth bothering.

There's no working grpc-lb-server to illustrate how to work with grpc-lb-client and grpc-lb-consul.

I am running the grpc-lb-consul and the grpc-lb-client cmds together with consul.

When the client executes the call to "Servers" at

FATAL rpc error: code = Internal desc = rpc error: code = Unavailable desc = grpc: the connection is unavailable

I debugged the problem and found, that the call to "Load" in order to update the score
https://github.com/bsm/grpclb/blob/master/balancer/backend.go#L63
fails.

I am not sure why the grpc-lb-client does not work but I assume, that the new balancer
created at https://github.com/bsm/grpclb/blob/master/cmd/grpc-lb-consul/main.go#L41 considers load reporting if nil configuration is used (and thus default applied)?

How is this supposed to work?

Why not use grpclb spec?