Flock-rs is a scalable message broker inspired by a talk from LinkedIn that allows real-time events to be published over WebSocket connections.

Flock-rs consists of two components: flight and traffic-control, which need to be runned together in order for the component to work.

Flight is the components that connects directly with end users over WebSocket connections. It allows user to connect to the component and subscribe or unsubscribe to a topic of interest. It can scales horizontally by running multiple instances adjacently.

Before running the service, you must have a traffic-control and a watchtower service already running.

To start flight service, go inside flight folder and execute

cargo run

By default, the the urls of the watchtower is "http://localhost:8088". However, it can be set using the environment variable WATCHTOWER_URLS. On the other hand, the url of traffic-control will be acquired using the watchtower as a service discovery directly.

Unfortunately, there is currently no official client written for flight yet. However, you may implement custom client using the followings:

To connect to a flight server simplies connect using WebSocket with the path of /ws

To subscribe, send the payload with the following format:

{
    "type": "Subscribe",
    "request_id": "[any_id]",
    "topic": "[your_topic]",
}

Then, you will receive the reply in the following format:

{
    "type": "response",
    "topic": "[your_topic]",
    "subscribed": true,
    "request_id": "[your_id]"
}

To unsubscribe, send the payload with the following format:

{
    "type": "Unsubscribe",
    "request_id": "[any_id]",
    "topic": "[your_topic]",
}

Then, you will receive the reply in the following format:

{
    "type": "response",
    "topic": "[your_topic]",
    "subscribed": false,
    "request_id": "[your_id]"
}

Traffic-control is a component that controls multiple flight instances. In order to publish events to end-user, you will need to publish to traffic-control.

Traffic-control requires Redis for shared storage.You may run the redis using the docker-compose.yml file.

docker-compose up -d

Be sure to comment out other services that are not needed.

Then, to run traffic-control,

cargo run

The library includes a Rust client. To include in your project, add the following to your Cargo.toml file.

traffic_control_client = { git = "https://github.com/warunyoud/flock-rs", branch = "main" }

The basic functionalities of the client can be described as followed:

use traffic_control_client::{TrafficControlClient, Error};

const USERNAME: &str = "admin";
const PASSWORD: &str = "password";

async fn main() {
    let traffic_control_client = TrafficControlClient::new(USERNAME, PASSWORD);

    // To publish
    let base_url = "http://127.0.0.1:8080";
    let topic = "mytopic";
    let payload = "{ \"message\": \"hello\" }";
    traffic_control_client.publish(base_url, topic, payload).await.unwrap();
}

To install the python client,

pip install traffic-control-client

Unlike the Rust client, in order to keep the service on the registry, you will have to manually call the ping function.

from traffic_control_client import PyTrafficControlClient

traffic_control_client = PyTrafficControlClient("admin", "password")

# To publish
base_url = "http://127.0.0.1:8080"
topic = "mytopic"
payload = "{ \"message\": \"hello\" }"
traffic_control_client.publish(base_url, topic, payload) 

You may write your own client and make the appropriate http requests in order to publish events.

Authentication for WebSocket client is not yet implemented. Please wait for newer version in the future.