Code Monkey home page Code Monkey logo

rabbitmq-package's Introduction

RabbitMQ Package

This is a Kurtosis Starlark Package that allows you to spin up an n node RabbitMQ Cluster. It spins up 3 nodes by default but you can tweak it. An etcd instance is also started to manage clustering.

Run

This assumes you have the Kurtosis CLI installed.

Simply run

kurtosis run github.com/kurtosis-tech/rabbitmq-package

Both the management (default: 15672) and AMQP (default: 5672) ports are exposed. An administrator user "admin" is created with a default password set to "admin".

Configuration

Click to see configuration

You can configure this package using a JSON structure as an argument to the kurtosis run function. The full structure that this package accepts is as follows, with default values shown (note that the // lines are not valid JSON and should be removed!):

{
    // The number of nodes
    "rabbitmq_num_nodes": 3,

    // The image to run
    "rabbitmq_image": "rabbitmq:3-management",

    // The management interface port number
    "rabbitmq_management_port": 15672,

    // The AMQP interface port number
    "rabbitmq_amqp_port": 5672,

    // The administrator user name and password
    "rabbitmq_admin_user": "admin",
    "rabbitmq_admin_password": "admin",

    // The virtual host to create
    "rabbitmq_vhost": "test",

    // Additional environment variables that will be set on the container
    "rabbitmq_env_vars": {}
}

These arguments can either be provided manually:

kurtosis run github.com/kurtosis-tech/rabbitmq-package '{"rabbitmq_image":"rabbitmq:3-management"}'

or by loading via a file, for instance using the args.json file in this repo:

kurtosis run github.com/kurtosis-tech/rabbitmq-package --enclave rabbitmq "$(cat args.json)"

etcd

This package leverages the Kurtosis etcd Package to spin up an instance of etcd to manage the RabbitMQ clustering. Once the etcd instance and the RabbitMQ cluster are up, you can list the RabbitMQ keys stored in the etcd database.

$ etcdctl get --prefix /rabbitmq
/rabbitmq/discovery/rabbitmq/clusters/default/nodes/rabbit@0a16a671bb48
{"lease_id":7587869983329495568,"node":"rabbit@0a16a671bb48","ttl":61}
/rabbitmq/discovery/rabbitmq/clusters/default/nodes/rabbit@2dff34180bf4
{"lease_id":7587869983329495558,"node":"rabbit@2dff34180bf4","ttl":61}
/rabbitmq/discovery/rabbitmq/clusters/default/nodes/rabbit@6b14a7ccfa6d
{"lease_id":7587869983329495578,"node":"rabbit@6b14a7ccfa6d","ttl":61}

Using this in your own package

Kurtosis Packages can be used within other Kurtosis Packages, through what we call composition internally. Assuming you want to spin up RabbitMQ and your own service together you just need to do the following

main_rabbitmq_module = import_module("github.com/kurtosis-tech/rabbitmq-package/main.star")

# main.star of your RabbitMQ + Service package
def run(plan, args):
    plan.print("Spinning up the RabbitMQ Package")
    # this will spin up RabbitMQ and return the output of the RabbitMQ package [rabbitmq-node-0 .. rabbitmq-node-n]
    # any args (including num_nodes) parsed to your package would get passed down to the RabbitMQ Package
    rabbitmq_run_output = main_rabbitmq_module.run(plan, args)

rabbitmq-package's People

Contributors

h4ck3rk3y avatar laurentluce avatar peeeekay avatar

Stargazers

avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

laurentluce

rabbitmq-package's Issues

Clustering fails on K8S with an etcd lease error

This is seen on K8S. Need to run more tests.

2023-06-22 21:32:50.388407+00:00 [info] <0.6984.0> etcd peer discovery: successfully connected to etcd
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** State machine rabbitmq_peer_discovery_etcd_v3_client terminating
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** Last event = {info,#{event => 'KeepAliveHalted',
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                         lease_id => 7587871464148287793,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                         reason =>
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                             <<"etcdserver: requested lease not found">>}}
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** When server state  = {connected,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                             {statem_data,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 ["etcd:2379"],
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 [],rabbitmq_peer_discovery_etcd_v3_client,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 <0.6985.0>,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 #Ref<0.2393751246.1218445315.205276>,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 <<"rabbitmq">>,<<"default">>,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 7587871464148287793,61,<0.6987.0>,70,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>                                 undefined,undefined}}
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** Reason for termination = error:function_clause
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** Callback modules = [rabbitmq_peer_discovery_etcd_v3_client]
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** Callback mode = [state_functions,state_enter]
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> ** Stacktrace =
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0> **  [{rabbitmq_peer_discovery_etcd_v3_client,connected,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>          [info,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>           #{event => 'KeepAliveHalted',lease_id => 7587871464148287793,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>             reason => <<"etcdserver: requested lease not found">>},
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>           {statem_data,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>               ["etcd:2379"],
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>               [],rabbitmq_peer_discovery_etcd_v3_client,<0.6985.0>,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>               #Ref<0.2393751246.1218445315.205276>,<<"rabbitmq">>,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>               <<"default">>,7587871464148287793,61,<0.6987.0>,70,undefined,
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>               undefined}],
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>          [{file,"rabbitmq_peer_discovery_etcd_v3_client.erl"},{line,174}]},
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>      {gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1426}]},
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]
2023-06-22 21:37:15.755876+00:00 [error] <0.6984.0>
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>   crasher:
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     initial call: rabbitmq_peer_discovery_etcd_v3_client:init/1
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     pid: <0.6984.0>
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     registered_name: rabbitmq_peer_discovery_etcd_v3_client
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     exception error: no function clause matching
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                      rabbitmq_peer_discovery_etcd_v3_client:connected(info,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                       #{event =>
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                          'KeepAliveHalted',
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                         lease_id =>
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                          7587871464148287793,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                         reason =>
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                          <<"etcdserver: requested lease not found">>},
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                       {statem_data,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        ["etcd:2379"],
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        [],
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        rabbitmq_peer_discovery_etcd_v3_client,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        <0.6985.0>,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        #Ref<0.2393751246.1218445315.205276>,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        <<"rabbitmq">>,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        <<"default">>,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        7587871464148287793,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        61,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        <0.6987.0>,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        70,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        undefined,
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>                                                                        undefined}) (rabbitmq_peer_discovery_etcd_v3_client.erl, line 174)
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>       in function  gen_statem:loop_state_callback/11 (gen_statem.erl, line 1426)
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     ancestors: [rabbitmq_peer_discovery_etcd_sup,<0.713.0>]
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     message_queue_len: 0
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     messages: []
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     links: [<0.714.0>]
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     dictionary: []
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     trap_exit: false
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     status: running
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     heap_size: 10958
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     stack_size: 28
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>     reductions: 31870
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>   neighbours:
2023-06-22 21:37:15.756372+00:00 [error] <0.6984.0>
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>     supervisor: {local,rabbitmq_peer_discovery_etcd_sup}
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>     errorContext: child_terminated
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>     reason: {function_clause,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                 [{rabbitmq_peer_discovery_etcd_v3_client,connected,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                      [info,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                       #{event => 'KeepAliveHalted',
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                         lease_id => 7587871464148287793,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                         reason => <<"etcdserver: requested lease not found">>},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                       {statem_data,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                           ["etcd:2379"],
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                           [],rabbitmq_peer_discovery_etcd_v3_client,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                           <0.6985.0>,#Ref<0.2393751246.1218445315.205276>,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                           <<"rabbitmq">>,<<"default">>,7587871464148287793,61,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                           <0.6987.0>,70,undefined,undefined}],
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                      [{file,"rabbitmq_peer_discovery_etcd_v3_client.erl"},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                       {line,174}]},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                  {gen_statem,loop_state_callback,11,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                      [{file,"gen_statem.erl"},{line,1426}]},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                  {proc_lib,init_p_do_apply,3,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                      [{file,"proc_lib.erl"},{line,240}]}]}
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>     offender: [{pid,<0.6984.0>},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {id,rabbitmq_peer_discovery_etcd_v3_client},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {mfargs,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                    {rabbitmq_peer_discovery_etcd_v3_client,start_link,
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                        [#{endpoints => ["etcd:2379"]}]}},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {restart_type,permanent},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {significant,false},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {shutdown,infinity},
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>                {child_type,worker}]
2023-06-22 21:37:15.757246+00:00 [error] <0.714.0>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.