Code Monkey home page Code Monkey logo

psrabbitmq's Introduction

PSRabbitMQ

PowerShell module to send and receive messages from a RabbitMQ server.

All credit to @gpduck, all blame for butchering to @ramblingcookiemonster.

Functionality

Send and receive messages through a RabbitMQ server:

Send and receive

Listen for RabbitMQ messages until you break execution:

Listener gif

Instructions

Prerequisites

Managing RabbitMQ with RabbitMQTools

RabbitMQTools is a separate module for managing RabbitMQ over the REST API. It was originally written by @mariuszwojcik, with slight modifications from @ramblingcookiemonster.

Skip this section if you're just interested in using PSRabbitMQ to send and receive messages.

# Install the module
    Install-Module RabbitMQTools

# No PowerShellGet module?
    # Download RabbitMQTools
    # https://github.com/RamblingCookieMonster/RabbitMQTools/archive/master.zip
    # Unblock the archive
    # Copy the RabbitMQTools module to one of your module paths ($env:PSModulePath -split ";")

#Import the module
    Import-Module RabbitMQTools -force

#Get commands from the module
    Get-Command -module RabbitMQTools

#Get help for a command
    Get-Help Get-RabbitMQOverview

#Define some credentials.  You need an account on RabbitMQ server before we can do this
    $credRabbit = Get-Credential
 
#Convenience - tab completion support for BaseUri
    Register-RabbitMQServer -BaseUri "https://rabbitmq.contoso.com:15671"
 
#I don't want to keep typing those common parameters... we'll splat them
    $Params = @{
        BaseUri = "https://rabbitmq.contoso.com:15671"
        Credential = $credRabbit
    }
 
#Can you hit the server?
    Get-RabbitMQOverview @params
 
#This shows how to create an Exchange and a Queue
#Think of the Exchange as the Blue USPS boxes, and a queue as the individual mailboxes the Exchanges route messages to
    $ExchangeName = "TestFanExc"
    $QueueName = 'TestQueue'
 
#Create an exchange
    Add-RabbitMQExchange @params -name $ExchangeName -Type fanout -Durable -VirtualHost /
 
#Create a queue for the exchange - / is a vhost initialized with install
    Add-RabbitMQQueue @params -Name $QueueName -Durable -VirtualHost /
 
#Bind them
    Add-RabbitMQQueueBinding @params -ExchangeName $ExchangeName -Name $QueueName -VirtualHost / -RoutingKey TestQueue
 
#Add a message to the exchange
    $message = [pscustomobject]@{samaccountname='cmonster';home='\\server\cmonster$'} | ConvertTo-Json
    Add-RabbitMQMessage @params -VirtualHost / -ExchangeName $ExchangeName -RoutingKey TestQueue -Payload $Message
 
#View your changes:
    Get-RabbitMQExchange @params
    Get-RabbitMQQueue @params
    Get-RabbitMQQueueBinding @params -Name $QueueName
 
#View the message we added:
    Get-RabbitMQMessage @params -VirtualHost / -Name $QueueName
    <#
 
        # = the number in the queue
        Queue = name of the queue
        R = whether we've read it (blank when you first read it, * if something has read it)
        Payload = your content.  JSON is helpful here.
 
              # Queue                R Payload
            --- -----                - -------
              1 TestQueue            * {...
    #>
 
#View the payload for the message we added:
    Get-RabbitMQMessage @params -VirtualHost / -Name $QueueName | Select -ExpandProperty Payload

    <#
        JSON output:

        {
            "samaccountname":  "cmonster",
            "home":  "\\\\server\\cmonster$"
        }
    #>

#Example processing the message
    $Incoming = Get-RabbitMQMessage @params -VirtualHost / -Name $QueueName -count 1 -Remove
    $IncomingData = $Incoming.payload | ConvertFrom-Json
    #If something fails, add the message back, or handle with other logic...

    #It's gone
    Get-RabbitMQMessage @params -VirtualHost / -Name $QueueName -count 1
 
    #We have our original data back...
    $IncomingData
 
    #There are better ways to handle this, illustrative purposes only : )
 
#Remove the Queue
    Remove-RabbitMQQueue @params -Name $QueueName -VirtualHost /
 
#Remove the Exchange
    Remove-RabbitMQExchange @params -ExchangeName $ExchangeName -VirtualHost /
 
#Verify that the queueu and Exchange are gone:
    Get-RabbitMQExchange @params
    Get-RabbitMQQueue @params

PSRabbitMQ

This is a module for sending and receiving messages using a RabbitMQ server and the .NET client library. Originally written by CD, slight modification by @ramblingcookiemonster.

# Install the module
    Install-Module PSRabbitMQ

# No PowerShellGet module?
    # Download PSRabbitMQ
    # https://github.com/RamblingCookieMonster/PSRabbitMQ/archive/master.zip
    # Unblock the archiveiles
    # Copy the PSRabbitMQ module folder to one of your module paths ($env:PSModulePath -split ";")

#Import the module
    Import-Module PSRabbitMQ

#List commands in PSRabbitMQ
    Get-Command -Module PSRabbitMQ

#Get help for a function in PSRabbitMQ
    Get-Help Send-RabbitMQMessage -Full

#Define a default RabbitMQ server and get a credential to use
    Set-RabbitMQConfig -ComputerName rabbitmq.contoso.com
    $CredRabbit = Get-Credential

#Set some common parameters we will always use:
    $Params = @{
        Credential = $CredRabbit
        Ssl = 'Tls12' #I'm using SSL... omit this if you aren't
    }

#Assumes an exchange and bound queue set up per RabbitMQTools example:
    #$ExchangeName = "TestFanExc"
    #$QueueName = 'TestQueue'

#Start waiting for a RabbitMQ message for 120 seconds
    $Incoming = Wait-RabbitMQMessage -Exchange TestFanExc -Key 'TestQueue' -QueueName TestQueue -Timeout 120 @Params

#Open a new PowerShell Window import PSRabbitMQ, and send a persistent message
    Send-RabbitMQMessage -Exchange TestFanExc -Key 'TestQueue' -InputObject "Hello!" -Persistent @Params

#Send an arbitrary object
    $SomeObject = [pscustomobject]@{
        Some='Random'
        Data = $(Get-Date)
    }

    Send-RabbitMQMessage -Exchange TestFanExc -Key 'TestQueue' -InputObject $SomeObject -Persistent -Depth 2 @Params

        <#
            # InputObject is serialized when sent,
            # deserialized on the receiving end.
            # No need for messing with JSON

            Some   Data
            ----   ----
            Random 6/24/2015 4:24:51 PM
        #>

Initial changes

Temporary section to document changes since reciept of code. Will retire this eventually and rely on git commits.

  • 2015/6/23

    • Added option for SSL connections
    • Added option for authentication
    • Created public New-RabbitMQConnectionFactory function to simplify handling the new options
    • Created Add-RabbitMQConnCred private function to extract username/password from cred and add to factory
    • Created New-RabbitMQSslOption private function to simplify setting SSL options.
      • Note: the CertPath/CertPhrase/AcceptablePolicyErrors aren't specified by any calls to the function. Have not tested these.
    • Renamed private parse function to ConvertFrom-RabbitMQDelivery, made it public. Allows parsing from Register-RabbitMQEvent.
    • Wasn't sure how these were being used. Added handling for specifying an existing queue name and associated details (e.g. durable)
    • Converted timeouts to seconds
    • Added a LoopInterval (seconds) parameter for dequeue timeout
    • Added comment based help
    • Made asinine changes to formatting and organization. Sorry!
    • Wrote no new tests. Sorry!
  • 2015/6/24

    • Replaced client dll with latest bits
    • Resolved issue with credential handling due to dll changes
    • Added config handling for computername (get/set rabbitmqconfig) on appropriate functions
    • Added persistent option for sending messages

Notes

I don't know what messaging is and I'm terrible with code. Apologies for ugly, inefficient, or broken stuff : )

TODO:

  • Break down functions a bit more. For example, offer functions to handle acknowledgements. I might retrieve a message requireing acknowledgement, and only send the ack down the line if my code meets certain criteria.

References:

psrabbitmq's People

Contributors

chrislynchhpe avatar gaelcolas avatar gpduck avatar lukaszgi avatar powershellninja avatar ramblingcookiemonster avatar th0sxph avatar yuvalmarciano avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

psrabbitmq's Issues

Question about a "retrieve loop" to drain a queue with multiple items in it.

If I have a queue with 3 items in it, should this drain the queue and display each item?

$maxLoops = 5 
For ($i=0; $i -le $maxLoops; $i++) 
{
    $Incoming = Wait-RabbitMqMessage -Exchange $Exchange -Key $QueueName -QueueName $QueueName -Timeout 20 -vhost base @Params
    Write-Host $Incoming 
}

It pauses 20 seconds, then the Write-Host statement displays only the first item in the queue
From the RabbitMQ web interface, I see the queue empty after running this.

Thanks,
Neal

Stored Message using PSRabbitMq has Powershell wrapper XML when reading

Without the details of the security and servername, I'm sending a message like this:

$Exchange = "D.Data.Integrations.Topic"
$RoutingKey = "D.Data.Echo.BSSR.Legacy204Outbound"
$message = get-content "d:\Tests\EDI204_from_WCFSQLPollingEDIStaging_CTII_Original.xml"
Write-Host "Message = $message" 
Send-RabbitMQMessage -Exchange $Exchange -Key $RoutingKey -InputObject $message -vhost base -Persistent @Params
Write-Host "Sent Message to $RoutingKey "

When I read it using PSRabbitMQ it looks okay. But when doing a Get Message from RabbitMQ web interface (or BizTalk Server) the message has been wrapped with a lot of XML.

Example XML:

<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
  <Obj RefId="0">
    <TN RefId="0">
      <T>System.Object[]</T>
      <T>System.Array</T>
      <T>System.Object</T>
    </TN>
    <LST>
      <Obj RefId="1">
        <S>&lt;EGL_204Out_Sql xmlns="http://EGL.BizTalk.Common.Schemas.EGL_204Out_SQL"&gt;  </S>
        <MS>
          <S N="PSPath">D:\GitLTLShipmentOut_Dev\Echo.BSSR.Gateway.LTLShipmentOut\Echo.BSSR.Gateway.LTLShipmentOut.Transforms\Tests\EDI204_from_WCFSQLPollingEDIStaging_CTII_Original.xml</S>
          <S N="PSParentPath">D:\GitLTLShipmentOut_Dev\Echo.BSSR.Gateway.LTLShipmentOut\Echo.BSSR.Gateway.LTLShipmentOut.Transforms\Tests</S>
          <S N="PSChildName">EDI204_from_WCFSQLPollingEDIStaging_CTII_Original.xml</S>
          <Obj N="PSDrive" RefId="2">
            <TN RefId="1">
              <T>System.Management.Automation.PSDriveInfo</T>
              <T>System.Object</T>
            </TN>
            <ToString>D</ToString>
            <Props>
              <S N="CurrentLocation"></S>
              <S N="Name">D</S>
              <Obj N="Provider" RefId="3">
                <TN RefId="2">
                  <T>System.Management.Automation.ProviderInfo</T>
                  <T>System.Object</T>
                </TN>
                <ToString>Microsoft.PowerShell.Core\FileSystem</ToString>
                <Props>

I find my data in there, for example I have a city name of Ontario.
In my XML:
<N401>ONTARIO</N401>

In the above XML:

      <Obj RefId="102">
        <S>_x0009__x0009__x0009__x0009__x0009_&lt;N401&gt;ONTARIO RDC&lt;/N401&gt;</S>
        <MS>
          <S N="PSPath">D:\GitLTLShipmentOut_Dev\Echo.BSSR.Gateway.LTLShipmentOut\Echo.BSSR.Gateway.LTLShipmentOut.Transforms\Tests\EDI204_from_WCFSQLPollingEDIStaging_CTII_Original.xml</S>
          <S N="PSParentPath">D:\GitLTLShipmentOut_Dev\Echo.BSSR.Gateway.LTLShipmentOut\Echo.BSSR.Gateway.LTLShipmentOut.Transforms\Tests</S>
          <S N="PSChildName">EDI204_from_WCFSQLPollingEDIStaging_CTII_Original.xml</S>
          <Ref N="PSDrive" RefId="2" />
          <Ref N="PSProvider" RefId="3" />
          <I64 N="ReadCount">98</I64>
        </MS>
      </Obj>

How do I get the message to be stored in my simple original XML without all that Powershell XML wrapper "stuff" around it?

Thanks,
Neal Walters

Wait-RabbitMqMessage drains queue with one run

Hello,

I'm using the module and when I run Wait-RabbitMqMessage it removes all messages but only returns 1 output. Trying to understand if this is the expected behavior as I expected it to be more like the Get-RabbitMQMessage where it would return 1 object per run as Wait-RabbitMqMessage doesn't have the count parameter.

Get-NetIPAddress | Select-Object -ExpandProperty IPAddress | Send-RabbitMQMessage -Exchange TestFanExc -Key 'TestQueue'-Persistent @Params -SerializeAs "text/plain"

$Incoming = Wait-RabbitMQMessage -Exchange TestFanExc -Key 'TestQueue' -QueueName TestQueue2 -Timeout 120 @Params
$Incoming
# ::1

Thank you,
Kensel

Wait-rabbitmqMessage Timeout question/issue

Hi,

I haven't tested this yet, but it looks like the While loop is only decreasing a counter [int], without pausing in that loop.
Also the $Timeout variable has a default value of 1, so $SecondsRemaining = [Double]::PositiveInfinity is never reached/used.

I would prefer a loop with a timespan like this: https://mjolinor.wordpress.com/2012/01/14/making-a-timed-loop-in-powershell/

But ensuring backward compatibility with an [int] as param requires a bit extra work and I don't have time atm.

So just filing for later.

No connection created when using plaintext port 5672

Attempting to leverage the library and connect to a internal testing instance of RabbitMQ and noticed via TCPDump that the client was not making the connection. It appears in the connection factory the port is being overridden to use the SSL port, even if SSL is not being specified:

https://github.com/RamblingCookieMonster/PSRabbitMq/blob/master/PSRabbitMq/New-RabbitMqConnectionFactory.ps1#L71-L72

$TcpPortProp = [RabbitMQ.Client.ConnectionFactory].GetField("Port")
$TcpPortProp.SetValue($Factory, 5671)

Perhaps it should be something like:

$TcpPortProp = [RabbitMQ.Client.ConnectionFactory].GetField("Port")
if( $Ssl ) {
  $TcpPortProp.SetValue($Factory, 5671)
} else {
  $TcpPortProp.SetValue($Factory, 5672)
}

I'll try to get something working and submit a PR but I'm not familiar with the .Net client DLL thats being leveraged here. I'm also unsure how to handle this bigger picture, ie: -Port flag for user overriding?

Complementary Module

Hi @RamblingCookieMonster

I have been using this module with great success on PS 5.1 so thank you for your hard work. I did however run into issues when trying to use on Linux in PS 7.0. I can go into detail on why but to suit my purposes (and as a learning experience) I have created a new module to complement PSRabbitMQ, focusing on Consuming.

It is yet very simple and a work in progress but I thought it might be worth sharing via the PSGallery. So far I have named it PSRabbitMq.Consumer as it is intended to be used with your module.

Would you be happy for me to publish such a module? As far as i know you haven't copyrighted the name but I felt that I would like to have your approval out of courtesy.

You can find my module here. All feedback and questions welcome.

Possible to connect to vhosts besides / ?

I have an exchange that is only accessible via a custom vhost. When I try to use the Send-RabbitMqMessage module to publish a message I get an error that the specified endpoint was not available.

The RabbitMQ logs shows the following error:

{amqp_error,access_refused, "access to vhost '/' refused for user 'mytest'", 'connection.open'}}

Is there any way to tell the Send-RabbitMQMessage module to connect to an alternate vhost?

Allow Empty String for -Exchange

Passing -Exchange an empty string should resort to the "default" exchange in RabbitMQ.

Taken from: https://www.rabbitmq.com/tutorials/tutorial-three-java.html

Nameless exchange

In previous parts of the tutorial we knew nothing about exchanges, but still were able to send messages to queues. That was possible because we were using a default exchange, which we identify by the empty string ("").

Recall how we published a message before:

channel.basicPublish("", "hello", null, message.getBytes());
The first parameter is the the name of the exchange. The empty string denotes the default or nameless exchange: messages are routed to the queue with the name specified by routingKey, if it exists.

Possibility to reduce parameter definitions

One thing I've been working on lately is DRY'ing out my PowerShell code, especially when it comes to defining parameters. Coming from Ruby, I can't stand how a PowerShell function isn't able to inherit parameter declarations from another base function. This feeling was unpleasant so I attempted to work around it. I noticed the code here has quite a few repeat parameters. This will allow you to define some common parameters and load them into a function:

https://gist.github.com/derekmwright/91504970eb876ca4ff9c6c5714ada2ef

It's still a little bit of a work in progress, but I figure more eyes is a good thing and maybe it can benefit. Esp w/ the chatter of a possible version bump.

How to reply to a message?

I'm implementing RPC via direct reply-to messages. The PHP part that sends a request works, I get the request in PowerShell, but I find nothing on how to send back an acknowledgement.

working variant 1:

while ($true) {
    Try {
        $data = Wait-RabbitMqMessage -Exchange $ExchangeName -Key $QueueName -QueueName $QueueName -Timeout 120 @Params -ErrorAction SilentlyContinue
        $data = $data |ConvertFrom-Json
        $data
    } Catch {
        # do nothing
        $_.Exception
    }
}

working variant 2:

Start-RabbitMqListener -ComputerName 'localhost' -Exchange $ExchangeName -QueueName $QueueName -Key $QueueName -Credential $credentials |% { $_ }

Both variants print out the message. But all I get is a string ... how do I send a response?

Bind queue with multiple keys when connecting to a channel for topic exchange routing

When Connecting a channel using Connect-RabbitMQChannel, the Key parameter is a unique [string] value, and the queue binding is done only once:

Connect-RabbitMqChannel.ps1 line 113

$Channel.QueueBind($QueueName, $Exchange, $Key)

When using Topic exchange, a great feature is to have a queue with multiple bindings based on different key, to have multiple routing: http://www.rabbitmq.com/tutorials/tutorial-five-dotnet.html
Although the extra bindings could be done later, I believe it would be nice to accept an array of keys, and bind for each routing key.

[edit]Was being dumb. Changing to [string[]] should be enough, will test tomorrow and create a new PR [/edit]

Register-RabbitMQEvent not working when not setting Creds as Param

I find out that there was a breaking change in a recent commit here:

if ($PSBoundParameters['Credential'])
{
$ArgList = $ComputerName, $Exchange, $Key, $Action, $Credential, $Ssl, $LoopInterval, $QueueName, $Durable, $Exclusive, $AutoDelete, $RequireAck,$prefetchSize,$prefetchCount,$global,[bool]$IncludeEnvelope
}
elseif ($PSBoundParameters['CertPath'])
{
$ArgList = $ComputerName, $Exchange, $Key, $Action, $CertPath, $CertPassphrase, $Ssl, $LoopInterval, $QueueName, $Durable, $Exclusive, $AutoDelete, $RequireAck,$prefetchSize,$prefetchCount,$global,[bool]$IncludeEnvelope
}

When Credential is not used as parameter, $ArgList is not set.

Explicit Declaration of Exchange in Connect-RabbitMQChannel

Hi,

I'm looking at addin an Active Declaration of basic exchanges (that is, non-autodelete, non-durable, and without any other fancy option) and would like your input.

The goal is to not need to create specific exchanges when consuming messages from a queue, so that we don't need to orchestrate the creation of the exchanges prior connecting a channel (No need to use RabbitMQTools before waiting for message for instance).
Looking at the simplest signature:
channel.ExchangeDeclare(exchangeName, ExchangeType.Direct);

I suggest that we:

  • only support this simple signature to avoid overloading the parameter sets
  • We do not Actively declare if the type is not sepecified (no default type, for backward compatibility and avoid errors)

Connect-RabbitMqChannel.ps1

#in param()
        [parameter(Mandatory = $false, ValueFromPipelineByPropertyName = $true)]
        [ValidateSet('Direct','Fanout','Topic','Headers')]
        [string]$ExchangeType = $null,

# [...]
#Actively declare the Exchange (as non-autodelete, non-durable)
        if($ExchangeType -and [string]::Empty -ne $Exchange) {
            $ExchangeResult = $Channel.ExchangeDeclare($Exchange,$ExchangeType)
        }

Here's the link to my branch for a potential PR.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.