Code Monkey home page Code Monkey logo

consul-cluster-manager's Introduction

Consul cluster manager for Vert.x ecosystem

Build Status

Introduction

Consul - based cluster manager that is plugable into Vert.x ecosystem. Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Motivation

As we all know Vert.x cluster managers are pluggable and so far we have 6 cluster manager implementations:

If you are already using Consul along with Vert.x within your system - it might be a good fit to use Consul directly as main manager for :

  • discovery and group membership of Vert.x nodes in a cluster Maintaining cluster wide topic subscriber lists (so we know which nodes are interested in which event bus addresses)
  • distributed map support;
  • distributed locks;
  • distributed counters;

Note : Cluster managers do not handle the event bus inter-node transport, this is done directly by Vert.x with TCP connections.

Implementation details

Current consul cluster manager implementation is fully based on vertx-consul-client and vertx-core.

How to use

Version compatibility matrix

Cluster manager Vert.x
0.1.0 3.6.0
0.2.0 3.6.1
0.3.0 3.6.2
0.4.0 3.6.3
1.0.0 3.7.0
1.1.0 3.7.1
1.2.0 3.8.0
1.2.1 3.8.1
1.2.2 3.8.2
3.9.1 3.9.1

Gradle

compile 'io.reactiverse:consul-cluster-manager:${cluster-manager-version}'

Maven

<dependency>
  <groupId>io.reactiverse</groupId>
  <artifactId>consul-cluster-manager</artifactId>
  <version>${cluster-manager-version}</version>
</dependency>

There's more than one way to create an instance of consul cluster manager.

  • Defaut one:

ConsulClusterManager consulClusterManager = new ConsulClusterManager(); // Consul agent should be running then on localhost:8500.

  • Excplicilty specifying configuration:
JsonObject options = new JsonObject()
.put("host", "consulAgentHost") // host on which consul agent is running, if not specified default host will be used which is "localhost".
.put("port", consulAgentPort) // port on wich consul agent is runing, if not specified default port will be used which is "8500".
/*
 * There's an option to utilize built-in internal caching.
 * @{Code false} - enable internal caching of event bus subscribers - this will give us better latency but stale reads (stale subsribers) might appear.
 * @{Code true} - disable internal caching of event bus subscribers - this will give us stronger consistency in terms of fetching event bus subscribers,
 * but this will result in having much more round trips to consul kv store where event bus subs are being kept.
 */
.put("preferConsistency", false)
/*
 * Option to provide custom service name which is visible on web ui
 */
.put("serviceName", "my-custom-service")
/*
 * There's also an option to specify explictly host address on which given cluster manager will be operating on.
 * By defult InetAddress.getLocalHost().getHostAddress() will be executed.
 * Linux systems enumerate the loopback network interface the same way as regular LAN network interfaces, but the JDK
 * InetAddress.getLocalHost method does not specify the algorithm used to select the address returned under such circumstances, and will
 * often return the loopback address, which is not valid for network communication.
 */
 .put("nodeHost", "10.0.0.1");
 // consul client options can be additionally specified as needed.
ConsulClusterManager consulClusterManager = new ConsulClusterManager(options);
  • Once cluster manager instance is created we can easily create clustered vertx. Voilà!
VertxOptions vertxOptions = new VertxOptions();
vertxOptions.setClusterManager(consulClusterManager);
Vertx.clusteredVertx(vertxOptions, res -> {
    if (res.succeeded()) {
	    // clustered vertx instance has been successfully created!
	    Vertx vertx = res.result();
	} else {
	    // something went wrong :(
	}
}

Restrictions

  • Compliant with ONLY Vert.x 3.6.0+ release.
  • The limit on a key's value size of any of the consul maps is 512KB.

consul-cluster-manager's People

Contributors

deedarb avatar dependabot[bot] avatar romalev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

consul-cluster-manager's Issues

Set custom service name

Option to provide custom service name would make more meaningful admin page of consul web ui.

We have multiple services, but when i open web ui, it is not possible to understand service's purpose without inspecting logs. Is there any limitations why variable SERVICE_NAME is constant?

EventBus address always get first one

I have three node( instance):
172.25.130.241: 4600, 172.25.130.241: 4601, 172.25.130.241: 4602

and register:
eb.consumer("abc-addr", ...)

each node fire event:
eb.request("abc-addr", data, ...)

but, it always the newest one process the message. (ie, 172.25.130.241: 4602)
i thank the "ChoosableIterable" implement class "ChoosableSet" or "ConsulAsyncMultiMap"
should keep previous pointer or instance for Load-Balance chooser

NullPointerException on startup

Consul 1.6.2 is running in docker on localhost at port 8500, using all the defaults. I bootstrap per the docs;

    ConsulClusterManager consulClusterManager = new ConsulClusterManager();
    options.setClusterManager(consulClusterManager);

The cluster manager dies with a NPE;

[ERROR] i.v.c.i.ContextImpl - Unhandled exception
java.lang.NullPointerException: null
        at io.vertx.spi.cluster.consul.ConsulClusterManager.entryUpdated(ConsulClusterManager.java:478)
        at io.vertx.spi.cluster.consul.impl.ConsulMapListener.lambda$kvWatchHandler$0(ConsulMapListener.java:89)
        at io.vertx.ext.consul.impl.WatchImpl.sendSuccess(WatchImpl.java:213)
        at io.vertx.ext.consul.impl.WatchImpl.lambda$go$3(WatchImpl.java:183)
        at io.vertx.ext.consul.impl.WatchImpl.lambda$fetch$5(WatchImpl.java:198)
        at io.vertx.ext.consul.impl.WatchImpl$KeyPrefix.lambda$wait$1(WatchImpl.java:62)
        at io.vertx.ext.consul.impl.ConsulClientImpl.lambda$reqOnContext$61(ConsulClientImpl.java:731)
        at io.vertx.ext.web.client.impl.HttpContext.handleDispatchResponse(HttpContext.java:308)
        at io.vertx.ext.web.client.impl.HttpContext.execute(HttpContext.java:295)
        at io.vertx.ext.web.client.impl.HttpContext.next(HttpContext.java:270)
        at io.vertx.ext.web.client.impl.predicate.PredicateInterceptor.handle(PredicateInterceptor.java:69)
        at io.vertx.ext.web.client.impl.predicate.PredicateInterceptor.handle(PredicateInterceptor.java:32)
        at io.vertx.ext.web.client.impl.HttpContext.next(HttpContext.java:267)
        at io.vertx.ext.web.client.impl.HttpContext.fire(HttpContext.java:277)
        at io.vertx.ext.web.client.impl.HttpContext.dispatchResponse(HttpContext.java:238)
        at io.vertx.ext.web.client.impl.HttpContext.lambda$null$2(HttpContext.java:367)
        at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369)
        at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
        at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)

issue with "No handlers for address"

i have strange error with clustered event bus handler (using reactiverse/consul-cluster-manager and vertx 3.9.5)

io.vertx.core.eventbus.ReplyException: No handlers for address NotifierService
    at io.vertx.core.eventbus.impl.EventBusImpl.deliverMessageLocally(EventBusImpl.java:411)
    at io.vertx.core.eventbus.impl.EventBusImpl.deliverMessageLocally(EventBusImpl.java:360)
    at io.vertx.core.eventbus.impl.clustered.ClusteredEventBus.sendToSubs(ClusteredEventBus.java:347)
    at io.vertx.core.eventbus.impl.clustered.ClusteredEventBus.onSubsReceived(ClusteredEventBus.java:235)
    at io.vertx.core.eventbus.impl.clustered.ClusteredEventBus.lambda$sendOrPub$8(ClusteredEventBus.java:227)
    at io.vertx.core.impl.FutureImpl.dispatch(FutureImpl.java:105)
    at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:150)
    at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:111)
    at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:176)
    at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:21)
    at io.vertx.core.impl.SucceededFuture.onComplete(SucceededFuture.java:41)
    at io.vertx.core.Future.lambda$compose$3(Future.java:373)
    at io.vertx.core.impl.FutureImpl.dispatch(FutureImpl.java:105)
    at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:150)
    at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:111)
    at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:176)
    at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:21)
    at io.vertx.core.impl.FutureImpl.dispatch(FutureImpl.java:105)
    at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:150)
    at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:111)
    at io.vertx.spi.cluster.consul.impl.ConsulAsyncMultiMap.lambda$doGet$16(ConsulAsyncMultiMap.java:201)
    at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:327)
    at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
    at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) 
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) 
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) 
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) 
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 
    at java.lang.Thread.run(Thread.java:834) [?:?]

where should i dig to find cause?
asked for help in https://gitter.im/eclipse-vertx/vertx-users - no response :|
may be app sometimes leaves cluster and that causes exception, how can i prove/fix that?
is it somehow network related? consul and apps on same machine

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.