Code Monkey home page Code Monkey logo

wavemq's Introduction

WAVEMQ - Tiered message bus for WAVE 3

WAVEMQ is the successor to the syndication tier in BOSSWAVE 2. It provides publish/subscribe communication using WAVE security. In addition, WAVEMQ is designed to be tiered. This means the expected topology looks something like this:

Topology overview

There is a message router on reliable hardware, typically located in the cloud, that is called the "designated router". Then there are additional message routers at each site. If the Internet connection between the site and the designated router goes down, then the site router will continue to locally deliver messages. In addition it will queue up messages for delivery to the designated router when connectivity permits. Similarly, if a service goes down, the site router will queue messages for later delivery to the service.

Although WAVEMQ is best-effort and you may lose messages or receive duplicates, there is also some persistence in the message queues, so that if a site router loses power or if the designated router is rebooted, any large queues of messages will not be lost.

Getting started: set up a site router

If you want to deploy a new site router, the procedure is as follows:

Download a release from github and save it as /usr/local/bin/wavemq.

Create the configuration file /etc/wavemq/wavemq.toml with the following contents:

[WaveConfig]
  database = "/var/lib/wavemq/wave"
  # this is optional, but required if you want your site to operate with no internet
  defaultToUnrevoked = true

  [WaveConfig.storage]
    # This is the default HTTPS server
    [WaveConfig.storage.default]
    provider = "http_v1"
    url = "https://standalone.storage.bwave.io/v1"
    version = "1"

[QueueConfig]
  queueDataStore = "/var/lib/wavemq/queue"
  # This is one day in seconds
  queueExpiry = 86400
  # 10k items (it will hit 100MB first)
  subscriptionQueueMaxLength = 10000
  # 100MB
  subscriptionQueueMaxSize = 100
  # 100k items (it will hit 1GB first)
  trunkingQueueMaxLength = 100000
  # 1GB
  trunkingQueueMaxSize = 1000
  # 30 seconds
  flushInterval = 30

[LocalConfig]
  # the address to connect to as an agent
  listenAddr = "127.0.0.1:4516"

[PeerConfig]
  # the address to connect to as a peer (not used for site router)
  listenAddr = "127.0.0.1:4515"

[RoutingConfig]
  PersistDataStore = "/var/lib/wavemq/persist"
  # This will be created for you
  RouterEntityFile = "/etc/wavemq/router.ent"
  [[RoutingConfig.Router]]
    Namespace = "the namespace you are interacting with"
    Address = "the designated router address"

If you are using the XBOS WAVEMQ designated router, then the final three lines should be:

[[RoutingConfig.Router]]
  Namespace = "GyAlyQyfJuai4MCyg6Rx9KkxnZZXWyDaIo0EXGY9-WEq6w=="
  Address = "wavemq.xbos.io:4515"

Finally, create a systemd unit to run wavemq in the background. Write this to /etc/systemd/system/wavemq.service:

[Unit]
Description="WAVEMQ"

[Service]
Restart=always
RestartSec=30
ExecStart=/usr/local/bin/wavemq /etc/wavemq/wavemq.toml

[Install]
WantedBy=multi-user.target

You can now start wavemq with sudo sytemctl daemon-reload; sudo systemctl start wavemq

Getting started: using the site router

First you need to create an entity for the service that will be connecting to the site router:

wv mke -o service.ent --expiry 1y

Then grant permissions from your namespace to your service entity:

wv rtgrant --attester namespace.ent --subject service.ent --expiry 1y "wavemq:subscribe,publish,[email protected]/*"

wv rtgrant --attester namespace.ent --subject service.ent --expiry 1y "wave:[email protected]/*"

Then you follow the example and fill in the namespace hash and entity file you are using. If you run the example, it should print out hello world five times then exit.

Getting started: creating a designated router

We assume you already have WAVE set up and running. To create a designated router, you first need to create a namespace:

wv mke -o namespace.ent --expiry 1y

Then you need to create an entity for your designated router:

wv mke -o router.ent --expiry 1y

Then you need to grant your router the permission to route on the namespace:

wv rtgrant --attester namespace.ent --subject router.ent --expiry 1y "wavemq:[email protected]/*" 

Finally, you need to create the proof that the router will hand to peers:

wv rtprove --subject router.ent -o routerproof.pem "wavemq:[email protected]/*"

Copy router.ent to /etc/wavemq/router.ent on the designated router. Also copy routerproof.pem to /etc/wavemq/routerproof.pem. Finally, to get the hash of the namespace for use in the config files, do:

wv inspect namespace.ent 

Which should give you something like:

= Entity
      Hash: GyD0mVNZxmMcL5bFSWgZ59SrMYPcTZuJpXrXH3zY4wN4Xw==
   Created: 2018-09-20 15:24:54 -0700 PDT
   Expires: 2058-09-10 14:24:54 -0800 PST
  Validity:
   - Valid: true
   - Expired: false
   - Malformed: false
   - Revoked: false
   - Message: 

That hash will need to appear in site router config files. Finally, you can now create the config file for the designated router /etc/wavemq/wavemq.toml with the following:

[WaveConfig]
  database = "/var/lib/wavemq/wave"
  defaultToUnrevoked = true

  [WaveConfig.storage]
    # This is the default HTTPS server
    [WaveConfig.storage.default]
    provider = "http_v1"
    url = "https://standalone.storage.bwave.io/v1"
    version = "1"

[QueueConfig]
  queueDataStore = "/var/lib/wavemq/queue"
  # This is one day in seconds
  queueExpiry = 86400
  # 1000 items
  subscriptionQueueMaxLength = 10000
  # 100MB
  subscriptionQueueMaxSize = 100
  # 10k items
  trunkingQueueMaxLength = 100000
  # 1GB
  trunkingQueueMaxSize = 3000
  # 30 seconds
  flushInterval = 30

[LocalConfig]
  # bind this to localhost to prevent clients from connecting directly.
  # they must only connect through a site router
  listenAddr = "127.0.0.1:4516"

[PeerConfig]
  listenAddr = "0.0.0.0:4515"

[RoutingConfig]
  PersistDataStore = "/var/lib/wavemq/persist"
  RouterEntityFile = "/etc/wavemq/router.ent"
  DesignatedNamespaceFiles = [
    "/etc/wavemq/routerproof.pem",
  ]

Create the systemd unit file as in the site router section, and start the service.

wavemq's People

Contributors

gtfierro avatar immesys avatar ddreyer avatar creachadair avatar samkumar avatar

Watchers

 avatar James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.