undeadlabs / discovery Goto Github PK

Is there a way to get a callback for when a node of a given service has been restored? For example:

Node A is running service test_service
Node B starts and is Polling for nodes of type test_service
Node B sees Node A and connects via the Discovery.Handler.NodeConnect

Node A goes down (elixir app not consul)
Node B does not get a callback
Node A returns
Node B is not getting a callback (however the: [info] Connected to: name@route is printing so they are reconnecting)

Example code for Node B:

defmodule Sample do
    use Application
    require Logger

    def start(_type, _args) do
        import Supervisor.Spec, warn: false
        children = 
        [
            worker(Discovery.Poller, 
                ["test_service", [Discovery.Handler.NodeConnect, &discoveryCB/1]], 
                id: :test_service_id)
        ]
        opts = [strategy: :one_for_one, name: Sample.Supervisor]
        Supervisor.start_link(children, opts)
    end

    def discoveryCB(services) do
        Logger.debug("[Sample] services available = #{inspect services}")
    end
end

While getting a callback for when the Node goes down is not really a big deal, getting a callback for when a node returns would be VERY helpful.

NOTE: If both consul and elixir go down, callbacks are hit upon the return of the service.

While using discovery for my application, it would seem that dead service instances are not getting removed.

I had two of my service running:

$ consul members
host1.example.com 192.168.0.100 alive client 0.3.0 2
host2.example.com 192.168.0.101 alive client 0.3.0 2
# other members ...
$ iex
iex> Discovery.Directory.services
%{"my_app" => #HashSet<[:"[email protected]",
   :"[email protected]"]>}

Host1 died and I forced it to leave the cluster with:

$ consul force-leave host1.example.com
$ consul members
host1.example.com 192.168.0.100 left client 0.3.0 2
host2.example.com 192.168.0.101 alive client 0.3.0 2
# other members ...

Unfortunately, at this point Discovery has not realized that the service on host1 is no longer available:

$ iex
iex> Discovery.Directory.services
%{"my_app" => #HashSet<[:"[email protected]",
   :"[email protected]"]>}

The Discovery.Directory process will sometimes miss registration messages if the node has come online but is not yet ready to receive messages. This will cause the directory to not contain the node even though the node is connected resulting in Discovery.select/3 from returning a :no_servers error.

Hi

Considering using this for a larger application

Should it be considered production-ready?

Hi all,

I was playing round with discovery and one thing that would be awesome is an example project with all the modules defined but stubbed out (kind of like how the cowboy team setup their examples https://github.com/ninenines/cowboy/tree/master/examples).

Thanks!

I am not sure, but seems to me that the heartbeat_ttl in the example is actually in milliseconds, not in seconds:

@heartbeat_ttl 10

should be

@heartbeat_ttl 10000

I've recently updated to Consul 0.5.0 on my dev machines and am having trouble getting the NodeConnect handler to work. I believe it might be related to a change with tags in Consul that is throwing this error on my agent.

Service tag "opt_name:my_app@my_machine" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes.

Curious if this is causing my issue.