Code Monkey home page Code Monkey logo

go-event-hub's Introduction

Go Event Hub

This is an Azure Event Hub AMQP 1.0 connector for the Go programming language (Golang) based on Apache Qpid Proton (an AMQP 1.0 C library).

GoDoc Build Status Go Report Card

Installation and setup

You can follow the procedure described in the Ansible script to compile and install Apache Qpid Proton on a Linux Debian machine: setup-amqp.yaml.

To install the Go wrapper (Electron) on a Linux Debian machine you can follow the procedure described in the docker file: oe-mseventhub.Dockerfile. For more details about those tweaks keep reading further down this document.

Requirements

  • Go 1.8
  • Apache Qpid Proton 0.19.0
  • Go Electron 0.19.0

Sample sender / receiver

There is a couple of Go files containing a main function to demonstrate the usage of the library. They could be use to send and receive messages via the Event Hub.

Connection details

The Go files expect to find some environment variables. Make sure you have configured the connection details of your Azure Event Hub e.g.:

export EH_TEST_NAMESPACE=foo
export EH_TEST_NAME=foo
export EH_TEST_SAS_POLICY_NAME=foo
export EH_TEST_SAS_POLICY_KEY=foo
export EH_TEST_CONSUMER_GROUP="\$Default"

Compile

To compile the Go apps:

  • for the sender: go build -o sender mainSenderApp.go
  • for the receiver: go build -o receiver mainReceiverApp.go

Run the sender

Note: the sender does not need EH_TEST_CONSUMER_GROUP to be setup. To run the sender: PN_TRACE_FRM=1 ./sender

Run the receiver

To run the receiver: PN_TRACE_FRM=1 ./receiver

For more details on the Proton environment variables check: https://qpid.apache.org/releases/qpid-proton-0.18.0/proton/c/api/group__transport.html

Tests

  • Make sure you set the Event Hub connection details as environment variables as previously explained.
  • Run all the tests: go test -v ./msauth/... ./eventhub/....

For a complete live end to end integration test involving a receiver configured with a partition offset as @latest, and a sender sending a message to the Event Hub, then listening to the receiver channel for the message previously sent take a look at end_to_end_integration_test.go. That also contain an end to end test scenario for a filter based on timestamps.

Docker

Base Proton/Electron image

  • Make sure the connection details of the Event Hub are exported as environment variables (as explained before).
  • To build the base image (and run the tests within within the building process): docker build -t oe-mseventhub --build-arg EH_TEST_NAMESPACE=$EH_TEST_NAMESPACE --build-arg EH_TEST_NAME=$EH_TEST_NAME --build-arg EH_TEST_SAS_POLICY_NAME=$EH_TEST_SAS_POLICY_NAME --build-arg EH_TEST_SAS_POLICY_KEY=$EH_TEST_SAS_POLICY_KEY -f oe-mseventhub.Dockerfile ..

Example of a sender

  • Build the docker image: docker build -t eh-sender_i -f sender.Dockerfile ..
  • Make sure the connection details of the Event Hub are exported as environment variables (as explained before).
  • Then run the docker container: docker stop eh-sender_c ; docker rm eh-sender_c ; docker run -d --name eh-sender_c -e EH_TEST_NAMESPACE=$EH_TEST_NAMESPACE -e EH_TEST_NAME=$EH_TEST_NAME -e EH_TEST_SAS_POLICY_NAME=$EH_TEST_SAS_POLICY_NAME -e EH_TEST_SAS_POLICY_KEY=$EH_TEST_SAS_POLICY_KEY -it eh-sender_i.
  • You could then log into the container: docker exec -it eh-sender_c bash, in there you can run the Go app like: ./sender (or PN_TRACE_FRM=1 ./sender).

Networking

To troubleshoot networking issues with Wireshark due to sockets, TLS connection, SSL keys, etc. you could use a filter like this: ip.addr == 192.168.XXX.YYY && (tcp.srcport == 5671 || tcp.dstport == 5671) where 5671 is the standard AMQP 1.0 TCP port to connect to the Azure Event Hub.

Make sure the Debug option of the Go instance representing an AMQP connection (eventhub.SenderOpts and eventhub.ReceiverOpts) is set to true so the SSL key is stored to a file to be later used by Wireshark to decrypt the network traffic capture. Check common.go for more details on this.

Further readings

AMQP / Azure Event Hub

TLS / Wireshark

go-event-hub's People

Contributors

jseely avatar tappoz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

tappoz jseely

go-event-hub's Issues

No Method to set Application Parameters on Event Hubs messages

Many applications of Event Hubs use Application Parameters to send metadata with events. There needs to be some method of attaching Application Parameters to the AMQP messages.

I have a naive implementation that changes the signature of the Send methods here #1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.