Comments (10)
adenhertog
commented on August 16, 2024
Hi Arjen,
Without seeing your exact code I'm going to make some assumptions here so
bear with me.
1. For something like an email handler, this sounds like a command handler
and so you should only define this once.
If you have defined the handler once but host it in two applications that
each own their own queue, then this will multiplex the message and hence
handle it in two places. In this case you'd need to decide which app
handles this command and only define the handler there.
2. You're correct in that if you send a message and no process is listening
then the message should be queued up. This will only happen if an
application queue has been subscribed to the message topic. This just means
you register a handler for the messages you're interested in and start the
app so @node-ts/bus can set all the topic/queue subscriptions. At this
point if you stop the app and send a message then it will be queued.
If you don't do this, then the message gets sent to a topic that has no
queue subscribers and so is discarded.
Hope this helps. If this isn't your situation then feel free to send a code
example and I can provide a more specific response.
Cheers
Andrew
…On Wed, 7 Oct 2020, 18:11 Arjen Haayman, ***@***.***> wrote:
Right now I'm missing options to more finegrained configuration of the
message queues. Some behaviour I'd like to change are:
-
When I'm running multiple instances of -say- an email-handler and I
notice that both handlers get the message resulting in multiple mails. How
to prevent this
-
When I send a message while there's no process listening to that
message it is discarded. Isn't the whole idea of message queues that
messages can sit in the queue patiently until they're handled? I'm quite
surprised about this behaviour and would like to have some options to
override this
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#92>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7K3W3NF5XHB2GLB7H3SJQPCLANCNFSM4SHBU4LA>
.
from bus.
haayman-imagem
commented on August 16, 2024
Hi Andrew,
The email handler was just an example. But what if I have some heavy load service that I want to have horizontally scaled? I'd like to be able to start up a bunch of handlers that each get assign a message round-robin. Isn't that a valid use case that should be possible?
from bus.
adenhertog
commented on August 16, 2024
Absolutely, and this should work out of the box. If you have a single
handler and you start that same node app several times then each should be
receiving from the same queue and provide the horizontal scalability.
Are you seeing something different?
…On Wed, 7 Oct 2020, 19:46 Arjen Haayman, ***@***.***> wrote:
Hi Andrew,
The email handler was just an example. But what if I have some heavy load
service that I want to have horizontally scaled? I'd like to be able to
start up a bunch of handlers that each get assign a message round-robin.
Isn't that a valid use case that should be possible?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7PDDGJH6OYX5KWTXB3SJQ2G5ANCNFSM4SHBU4LA>
.
from bus.
haayman-imagem
commented on August 16, 2024
That's just it: each instance gets the message but I'd like only one instance to get the message. I'm afraid I'm only a newby concerning message queueing but I expect that it's possible to configure that a message is sent to a single queue and the first handler pops the message from the queue so the other handlers don't get it.
What I saw was that a message got handled by multiple instances.
More info maybe: I was running the apps on different machines. The 'sending' app was using messageQueue name 'app1' and both receiving apps were using messageQueue name 'app2' (the real names are hidden for privacy reasons). Could this be the reason why both app2 apps were interfering with each other?
from bus.
haayman-imagem
commented on August 16, 2024
btw out of curiosity and highly offtopic: are you Dutch?
from bus.
adenhertog
commented on August 16, 2024
The setup you described sounds correct in order to get multiple instances
processing the one queue without duplication. Do you have a basic code
example I could have a look at? It might be easier to spot what's going on.
And yes I'm half Dutch. The surname is a dead giveaway :)
…On Wed, 7 Oct 2020, 20:02 Arjen Haayman, ***@***.***> wrote:
btw out of curiosity and highly offtopic: are you Dutch?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7NFCNY3T3GEDXG3YN3SJQ4FFANCNFSM4SHBU4LA>
.
from bus.
haayman-imagem
commented on August 16, 2024
it's not a problem that two apps share the same queue name? This morning I fixed a bug where app1 and app2 both used the same queue name and although both were connected I got the message there weren't any handlers so the messages got dropped (case 2 in the original post, so that's solved now). So I don't know what happens exactly when 2 apps use the same queueName but it appears that is causing troubles
from bus.
haayman-imagem
commented on August 16, 2024
I'm afraid my code is too complicated atm to serve as an example
from bus.
adenhertog
commented on August 16, 2024
Yes, each application needs its own dedicated queue. Each instance of an
application should share the same queue as it's other instances.
I hope I'm understanding your question correctly
…On Wed, 7 Oct 2020, 21:07 Arjen Haayman, ***@***.***> wrote:
it's not a problem that two apps share the same queue name? This morning I
fixed a bug where app1 and app2 both used the same queue name and although
both were connected I got the message there weren't any handlers so the
messages got dropped (case 2 in the original post, so that's solved now).
So I don't know what happens exactly when 2 apps use the same queueName but
it appears that is causing troubles
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7MCK5CGMLZEYNFV46DSJRDWXANCNFSM4SHBU4LA>
.
from bus.
valdestron
commented on August 16, 2024
Yes, each application needs its own dedicated queue. Each instance of an application should share the same queue as it's other instances. I hope I'm understanding your question correctly
…
On Wed, 7 Oct 2020, 21:07 Arjen Haayman, @.***> wrote: it's not a problem that two apps share the same queue name? This morning I fixed a bug where app1 and app2 both used the same queue name and although both were connected I got the message there weren't any handlers so the messages got dropped (case 2 in the original post, so that's solved now). So I don't know what happens exactly when 2 apps use the same queueName but it appears that is causing troubles — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#92 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOAR7MCK5CGMLZEYNFV46DSJRDWXANCNFSM4SHBU4LA .
What about postgres ? I have the same setup and postgres module is failing to save/retrieve data from the database, always erroring, because of race condition that happens when multiple containers handles messages and tries to save or upsert workflow data.
issue #134
from bus.
Related Issues (20)
- Construct workflows using container during mapping
- Port middleware over to master HOT 1
- RabbitMQ queue and exchange setup HOT 8
- The memory-queue is inappropriately deleting messages HOT 1
- Broken links in docs HOT 1
- Delay a retry by X seconds HOT 1
- Vulnerability in url-parse - dependency of amqplib HOT 1
- Rabbitmq transport polling HOT 2
- IoC with loopback HOT 4
- Named export <nameExport> not found HOT 2
- [bus-class-serializer]: package.json missing attributes HOT 1
- [Bug] Rabbitmq-transport.ts does not have reconnection if channel gets disconnected by some reasons
- How to setup DI container, when workflow needs the bus instance HOT 2
- Open to update `bus-sqs` to aws-sdk v3? HOT 1
- Support Mognodb Persistence for Workflows
- RabbitMQ transport doesn't allow custom message headers. HOT 4
- Problem with lodash and persisting workflow state
- Stuck workflows with error: "No existing workflow state found for message. Ignoring"
- [Feat] Support passing message and message context to Container Adapter HOT 1
- Persistent messages are not supported. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bus.