awslabs / amazon-sqs-java-temporary-queues-client Goto Github PK

We have implemented a maven library, in order to manage all the Amazon services, and a requestor-responder pattern implemented, where the requestor, is a Grails application, and responder is a Spring-boot application. Both applications, use the same maven library to communicate with Amazon (S3, SQS, Personalize,...).

On each requestor and responder restart, the number of temporary queues increase, and exist a lot of pending messages:

Reading the documentation, I saw that empty temporary queues must to be deleted, after 5 minutes idle. In order to test it, I purged all the messages on temporary queues, and after 30 minutos, those queues are not deleted:

What is the problem? We are using amazon-sqs-java-temporary-queues-client version 1.2.1.

I couldn't find the Ruby version of this. Is it still under development? Thank you in advance.

The VirtualQueuesClient currently only makes one heartbeat() call at the start of the receiveMessage() call.
https://github.com/awslabs/amazon-sqs-java-temporary-queues-client/blob/master/src/main/java/com/amazonaws/services/sqs/AmazonSQSVirtualQueuesClient.java#L347-L366

This means that if the max wait time specified in the ReceiveMessageRequest exceeds the ideal queue retention period, the underlying virtual queue will be deleted before the response arrives.

For longer running systems this introduces an issue where the user cannot specify an acceptably long IdleQueueRetentionPeriod (currently maxes out at 5min) and the heartbeat is only done once so the queue is deleted before responses arrive resulting in orphaned messages.

I did not find any Python implementation of the AmazonSQSTemporaryQueuesClient. Can you help?
Thank you.

The documentation says this is possible (last paragraph of https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-temporary-queues.html), but technically it's only possible on the underlying clients. The AmazonSQSTemporaryQueueClient currently hard-codes it:

https://github.com/awslabs/amazon-sqs-java-temporary-queues-client/blob/master/src/main/java/com/amazonaws/services/sqs/AmazonSQSTemporaryQueuesClient.java#L36

Either the documentation should be fixed or this should be exposed as a configuration parameter on the builder (which is my preference).

Are there any plans to provide a .net implement of this library?

Hi,

Do we have a NodeJs library for supporting temporary queues functionality.

Hi,
I did not find any Golang implementation for this. Can you help?
Thank you in advance.

Greetings!

I'm currently investigating how to implement an asynchronous request-response pattern using SQS/SNS, but with a few added wrinkles into the mix. When the API receives the initial request we want to forward it on to a separate service via SQS, perform some additional processing over a number of services (effectively lambdas, but not actual Lambdas) using SNS/SQS, and then send the response back to the API service via SQS. If this process takes longer than the API service is willing to wait (around 10 seconds), then we want it to fail-over to a separate queue so a different processor can pick it up and run with it.

This is a pretty complex use case, I know, and while I had initially hoped that this would work with this library, I'm struggling to figure out how to make this work. So I thought I'd run some of this by you guys and a) make sure my assumptions check out, and b) gather any suggestions on how I should approach this.

The biggest thing I've noticed is that the vast majority of the code assumes that the requester and responder are utilizing the same queue. I might be completely off track here, but I don't see any way to specify in the API two separate queues.

The second thing I've noticed is that the VirtualQueueClient doesn't seem to support multiple JVMs on the same queue, unless I'm really missing something. Specifically, there doesn't appear to be any logic for handling virtual queues created by other JVMs or processes; it looks like it just treats them as orphaned, since as far as I can tell it doesn't attempt to synchronize the virtual queues across hosts.

I've been working on drafting up a proof-of-concept that demonstrates this use case, but I'm starting to think I might need to modify the client code itself in order to support the concept of dual queues. That would at least let me use the original RequesterClient and ResponderClient libraries.

Is this a use case you guys support/are interested in supporting? I can continue investigating into this if so.

Currently developing an application using the request-response pattern.

After redeploying an application in ECS (old instance is stopped and new instance is started) The old instances queues remains in SQS and appears to get a few (sometimes 1 sometimes up too 11) of a strange message in it that i cant make sense of (".rO0ABXA=").

Because this is in the queue the sweeper doesn't end up deleting the old queue until the 4 days retention period is up and those messages get deleted.

In development environments and production environment this creates a lot of noise in the SQS console.

Would love if either this message didn't sent into the queue is not needed or if we were able to set the message retention attribute on the temporary queues that are made (at the moment its defaulting to 4 days and if I try to set the attribute for queue creation time then the virtual queue client throws illegal argument due to only allowing two specific attributes to be set.)

Would be great to have this in Rust too! We've got https://github.com/rusoto/rusoto and https://github.com/awslabs/aws-lambda-rust-runtime so far...

See #18

After look out the AWS SQS Temp queues documentation and code, I'm wondering why we need the temporary queues. I see that, virtual queues are being created on temporary queue url(HostQueue URL) using SQS temporary queue client. So, instead, can't we create the same virtual queues on the base queue?

I am creating an instance of the AmazonSQSTemporaryQueuesClient across all hosts in a fleet and stickying a single temporary SQS queue per host, which should not be deleted as long as the host is active. In my use-case, I am explicitly making a create queue request to initialize the host queue after creating the client in order to reduce latency of sendMessage calls since the host queues are created lazily otherwise. As a result, I am hitting the 5 TPS AWS account limit for SQS tagging because multiple hosts across the fleet are concurrently calling AmazonSQSIdleQueueDeletingClient#createQueue, which in turn makes tagQueue calls. I've already increased the idleQueueSweepingPeriod to a large number on the order of days to reduce the listQueueTags call volume. I understand this 5TPS is a soft limit but I think there are a few minimal changes to the AmazonSQSIdleQueueDeletingClient that can be made to reduce the TPS:

1. Add IDLE_QUEUE_RETENTION_PERIOD_TAG tag to CreateQueueRequest in AmazonSQSIdleQueueDeletingClient (link), instead of explicitly calling tagQueue. I am not sure if adding a tag via CreateQueueRequest contributes to the 5TPS limit but at the very least it can save a tagQueue API call.
2. Set a random initial delay value to the scheduled heartbeating to add some jitter.
3. This is an alternative to #2 - remove scheduled heartbeating after creating a new queue (or provide the option to do so) and rely on other API operations like receiveMessage to heartbeat the host queue. I'm not quite sure what other implications this might have as I am not too familiar with the code base.

Thanks!

@adam-aws Using amazon-sqs-java-temporary-queues-client for two way sqs communication. got this.

Sometimes, https://github.com/awslabs/amazon-sqs-java-temporary-queues-client/blob/master/src/main/java/com/amazonaws/services/sqs/util/SQSMessageConsumer.java#L120
will handle messages in SQSMessageConsumer-Thread; however, sometime ForkJoinPool.commonPool-worker will handle the messages, this will cause a java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read")

[ 02-02-2021 23:50:16.662 ] [ ForkJoinPool.commonPool-worker-0 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__82031328-9084-4353-8936-80908bf906a6: 90ebff84-c6f9-4c6c-bc65-86166d89074b [ ] Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read") (AccessControlContext.java:472) java.lang.RuntimeException: Exception encountered while processing message with ID ab66f646-19df-4c0d-a72c-38fe1218732c (SQSMessageConsumer.java:152) [ 02-02-2021 23:50:16.662 ] [ ForkJoinPool.commonPool-worker-2 ] [ ERROR ] [ SQSQueueUtils ] - Unexpected exception [ ] stackHash=6942cc2e68c98f5a7bbc792de5d39937 fullStackTrace=false Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read") (AccessControlContext.java:472) java.lang.RuntimeException: Exception encountered while changing message visibility with ID bc680709-72dc-4f6d-9ca8-78f8a96ba4a5 (SQSMessageConsumer.java:160) [ 02-02-2021 23:50:16.662 ] [ ForkJoinPool.commonPool-worker-0 ] [ ERROR ] [ SQSQueueUtils ] - Unexpected exception [ ] stackHash=1e8fa7c434f4edf19a7ab1f971d2eebc fullStackTrace=false Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read") (AccessControlContext.java:472) java.lang.RuntimeException: Exception encountered while processing message with ID bc680709-72dc-4f6d-9ca8-78f8a96ba4a5 (SQSMessageConsumer.java:152) [ 02-02-2021 23:50:16.662 ] [ ForkJoinPool.commonPool-worker-0 ] [ ERROR ] [ SQSQueueUtils ] - Unexpected exception [ ] stackHash=6942cc2e68c98f5a7bbc792de5d39937 fullStackTrace=false [ 02-02-2021 23:50:16.661 ] [ ForkJoinPool.commonPool-worker-0 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__0c148ce7-8e76-4fe3-85f2-f8cbfef2306e: bc680709-72dc-4f6d-9ca8-78f8a96ba4a5 [ ] [ 02-02-2021 23:50:16.661 ] [ ForkJoinPool.commonPool-worker-2 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__e0984e94-3419-460d-a09a-08a01f66c24b: ab66f646-19df-4c0d-a72c-38fe1218732c [ ] [ 02-02-2021 23:50:16.661 ] [ SQSMessageConsumer-Thread-3 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__d4d72115-90da-4324-8057-a6ae9d8a265b: 0d4b42fc-296d-42b4-ac7f-a708cf4f28f5 [ ] Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read") (AccessControlContext.java:472) java.lang.RuntimeException: Exception encountered while changing message visibility with ID ab258abf-9e1f-4d1b-b753-a2ea0d22eb50 (SQSMessageConsumer.java:160) [ 02-02-2021 23:50:16.592 ] [ ForkJoinPool.commonPool-worker-2 ] [ ERROR ] [ SQSQueueUtils ] - Unexpected exception [ ] stackHash=1e8fa7c434f4edf19a7ab1f971d2eebc fullStackTrace=false Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "com.amazonaws.sdk.enableRuntimeProfiling" "read") (AccessControlContext.java:472) java.lang.RuntimeException: Exception encountered while processing message with ID ab258abf-9e1f-4d1b-b753-a2ea0d22eb50 (SQSMessageConsumer.java:152) [ 02-02-2021 23:50:16.592 ] [ ForkJoinPool.commonPool-worker-2 ] [ ERROR ] [ SQSQueueUtils ] - Unexpected exception [ ] stackHash=6942cc2e68c98f5a7bbc792de5d39937 fullStackTrace=false [ 02-02-2021 23:50:16.592 ] [ ForkJoinPool.commonPool-worker-2 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__c6ddbd04-8d8e-439b-b7e7-16b84a934a7f: ab258abf-9e1f-4d1b-b753-a2ea0d22eb50 [ ] [ 02-02-2021 23:50:16.592 ] [ SQSMessageConsumer-Thread-3 ] [ WARN ] [ AmazonSQSVirtualQueuesClient ] - Orphaned message sent to __sql-fix-messaging-queue-ct8__ec6dc3a3-6a88-45fd-8db9-2852c4a1c211: 2a1ddb1e-3ba1-4698-89dc-3dfb71fa4e80 [ ]

Hello,
We ran into a scenario where we needed to Tag the virtual queues that are being created.
There should be a way to specify those Tags as part of the AmazonSQSRequesterClientBuilder which then applies those Tags to all the queues that it creates.

Based on the examples the following lines waits for the response

Message reply = sqs.sendMessageAndGetResponse(request, MAX_TIMEOUT, TimeUnit.SECONDS);

If the timeout occurs and the request fails, is the message in queue deleted? if not when will it get processed again?

I am trying to put temporary queues in place for bi-directional send and receive scenario where the producer is implemented using your library and the consumer is written manually in NodeJS since my lambda is based on JS.

Unfortunately though I have lots of problems with looping events like "Checking all queues begining with prefix" which seems to generate messages in the temporary queue which is really bad assuming it all costs money !

The console messages I receive in the producer:

2020-02-18 16:32:10.650 DEBUG 18099 --- [nsumer-Thread-1] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.650 DEBUG 18099 --- [onPool-worker-5] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.651 DEBUG 18099 --- [nsumer-Thread-1] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.651 DEBUG 18099 --- [onPool-worker-5] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.651 DEBUG 18099 --- [nsumer-Thread-1] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__cf226b82-ab67-4da2-9ac7-f3c372fe4baa,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.651 DEBUG 18099 --- [onPool-worker-5] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__66bb67da-1e84-43f8-a1a8-c3c1852dd6e5,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.651 DEBUG 18099 --- [nsumer-Thread-1] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: 3058cbdc6583f41873b1d3e14350f9a4
2020-02-18 16:32:10.651 DEBUG 18099 --- [onPool-worker-5] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: ccfa3e7753bb2b5e774a2b621bee72c4
2020-02-18 16:32:10.661 DEBUG 18099 --- [onPool-worker-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.661 DEBUG 18099 --- [onPool-worker-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.661 DEBUG 18099 --- [onPool-worker-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__931460ec-acfa-4cbd-95e4-faf482095a97,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.661 DEBUG 18099 --- [onPool-worker-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: c1cb0df2c89955f9454b08d1b9258f38
2020-02-18 16:32:10.693 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.693 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.693 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__cf226b82-ab67-4da2-9ac7-f3c372fe4baa,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: 3058cbdc6583f41873b1d3e14350f9a4
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__66bb67da-1e84-43f8-a1a8-c3c1852dd6e5,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: ccfa3e7753bb2b5e774a2b621bee72c4
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message body: .rO0ABXA=
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message body: 98e4c00964e43fd75ad2e2fa2f190887
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Message attribtues: {__AmazonSQSVirtualQueuesClient.QueueName={StringValue: __RequesterClientQueues__931460ec-acfa-4cbd-95e4-faf482095a97,StringListValues: [],BinaryListValues: [],DataType: String}}
2020-02-18 16:32:10.694 DEBUG 18099 --- [nsumer-Thread-3] c.a.s.sqs.MessageMD5ChecksumHandler      : Expected  MD5 of message attributes: c1cb0df2c89955f9454b08d1b9258f38
2020-02-18 16:32:10.699 DEBUG 18099 --- [nsumer-Thread-7] c.a.s.sqs.AmazonSQSVirtualQueuesClient   : Deleting Virtual Queue is 4 and Queue Name is https://sqs.eu-west-1.amazonaws.com/122333224379/__RequesterClientQueues__8e0145ad-8f87-4c15-8d55-733a96f950b2-0#__RequesterClientQueues__cf226b82-ab67-4da2-9ac7-f3c372fe4baa
2020-02-18 16:32:10.699 DEBUG 18099 --- [nsumer-Thread-5] c.a.s.sqs.AmazonSQSVirtualQueuesClient   : Deleting Virtual Queue is 4 and Queue Name is https://sqs.eu-west-1.amazonaws.com/122333224379/__RequesterClientQueues__8e0145ad-8f87-4c15-8d55-733a96f950b2-0#__RequesterClientQueues__931460ec-acfa-4cbd-95e4-faf482095a97
2020-02-18 16:32:10.699 DEBUG 18099 --- [nsumer-Thread-8] c.a.s.sqs.AmazonSQSVirtualQueuesClient   : Deleting Virtual Queue is 4 and Queue Name is https://sqs.eu-west-1.amazonaws.com/122333224379/__RequesterClientQueues__8e0145ad-8f87-4c15-8d55-733a96f950b2-0#__RequesterClientQueues__66bb67da-1e84-43f8-a1a8-c3c1852dd6e5


... skipped hundreds of more ...

Here is the producer code:

      try {
        com.amazonaws.services.sqs.model.SendMessageRequest request = new com.amazonaws.services.sqs.model.SendMessageRequest()
            .withMessageBody(objectMapper.writeValueAsString(domainEvent))
            .withQueueUrl(topic.getTopicURL());

        Message message = sqsRequester.sendMessageAndGetResponse(request, 20, TimeUnit.SECONDS);

        result.complete(objectMapper.readValue(message.getBody(), responseType));
      } catch (JsonProcessingException | TimeoutException e) {
        log.error("An error has occurred publishing domain event to SQS: {}", domainEvent, e);
      }

And the node JS consumer code:
If I understand correctly the consumer needs to answer to the request on the provided queue name for the response.

function sendSuccess(callback, response, queueUrl) {
  let params = {
    MessageBody: JSON.stringify(response),
    QueueUrl: queueUrl,
    MessageAttributes: {
      "__AmazonSQSVirtualQueuesClient.QueueName": {
        DataType: 'String',
        StringValue: queueUrl.split('#')[1]
      }
    }
  };

  sqs.sendMessage(params, function(sqsErr) {
    if(sqsErr) {
      console.log(`Problem answering to SQS event`, sqsErr);
      callback(sqsErr, response);
    } else {
      callback(null, response);
    }
  });
}

Please advise.
Best Regards
Alexander KIRILOV

Hi,
It looks like there is an invalid initialization of period and delay parameters in the following constructor of ScheduledSQSFutureTask:
After this.delay and this.period is set, the later code shall use this.delay and this.period.

        this.delay = unit.toNanos(delay);
        this.period = unit.toNanos(period);

        messageContent.setMessageAttributesEntry(DELAY_NANOS_ATTRIBUTE_NAME, 
                longMessageAttributeValue(**delay**));
        messageContent.setMessageAttributesEntry(PERIOD_NANOS_ATTRIBUTE_NAME, 
                longMessageAttributeValue(**period**));

        this.time = getTime(**delay**);

It looks it shall be

        this.delay = unit.toNanos(delay);
        this.period = unit.toNanos(period);

        messageContent.setMessageAttributesEntry(DELAY_NANOS_ATTRIBUTE_NAME, 
                longMessageAttributeValue(**this.delay**));
        messageContent.setMessageAttributesEntry(PERIOD_NANOS_ATTRIBUTE_NAME, 
                longMessageAttributeValue(**this.period**));

        this.time = getTime(**this.delay**);

Otherwise, the default of 5 minutes delay doesn't work and there is an infinite loop sending messages to check for Idle queues.

The maximum idle queue retention period is set to 5 min in the AmazonSQSIdleQueueDeletingClient ( https://github.com/awslabs/amazon-sqs-java-temporary-queues-client/blob/master/src/main/java/com/amazonaws/services/sqs/AmazonSQSIdleQueueDeletingClient.java#L73 ).

This upper bound appears to be arbitrary without a technical limitation keeping users from specifying a larger value.

Services with longer running tasks may not send/receive messages on the temporary queues as frequently as 5 minutes meaning that virtual queues can be unexpectedly deleted even when they are in use

I have setup 2 java components.

The first with an AmazonSQSRequester to invoke the sendMessageAndGetResponse method.
The seconds component with an SQSMessageConsumer and AmazonSQSResponder to invoke the sendResponseMessage method.

I am using amazon-sqs-java-temporary-queues-client version 2.0.0

It generally seems to work and I can send a message a get a response back however there is a background thread that throws runtime exceptions kind of randomly. They are all having some kind of issue performing a base64 decode. The sendMessageAndGetResponse method always completes without exception and I receive the expected responses but these runtime exceptions noted below happen most of the time however not all of the time and are not trappable with the surrounding try/catch. I've attached a file with code fragments which setup my sqsRequester and the method to send the message.

codefragments.txt

For example:
ERROR 36420 --- [nsumer-Thread-1] c.a.services.sqs.util.SQSQueueUtils : Unexpected exception

java.lang.RuntimeException: Exception encountered while processing message with ID 2d10d2c9-9b28-46ea-b4fb-758dd9dffc5c
at com.amazonaws.services.sqs.util.SQSMessageConsumer.handleMessage(SQSMessageConsumer.java:158) ~[amazon-sqs-java-temporary-queues-client-2.0.0.jar:na]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) ~[na:na]
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[na:na]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[na:na]
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661) ~[na:na]
at com.amazonaws.services.sqs.util.SQSMessageConsumer.poll(SQSMessageConsumer.java:122) ~[amazon-sqs-java-temporary-queues-client-2.0.0.jar:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.lang.IllegalArgumentException: Last unit does not have enough valid bits

Integrate CodeBuild for the integration tests

Upgrade tests to use JUnit 5 instead of JUnit 4

Hi. In testing, I keep running into the situation where AmazonSQSRequester creates hundreds of temporary queues, with hundreds of messages in each, and ends up in a state it seemingly can't recover from.

It's hard to figure out what's going on without getting way too well acquainted with the source code, but I've tried to produce a simple test case which can reliably reproduce the problem (or some problem anyway - hopefully not a different one):

	@Test
	public void test() throws InterruptedException {
		AmazonSQSRequesterClientBuilder
				.standard()
				.withAmazonSQS(AmazonSQSClientBuilder.defaultClient())
				.withInternalQueuePrefix("deleteme")
				.build();
		
		while (true) Thread.sleep(500);
	}

In this example, the queue deleteme does not yet exist at test start time. If I leave this test running for a while, after a few minutes, it starts spinning away, sucking significant CPU resources, and generating a huge volume of traffic to SQS (and messages sent to SQS) - here's a summarised extract of API activity over a three second period:

16:09:45.091 [SQSMessageConsumer-Thread-1] "Action=SendMessage&Version=2012
16:09:45.306 [SQSMessageConsumer-Thread-1] "Action=ListQueueTags&Version=20
16:09:45.518 [SQSMessageConsumer-Thread-1] "Action=SendMessage&Version=2012
16:09:45.558 [AmazonSQSIdleQueueDeletingClient-Thread-2] "Action=TagQueue&V
16:09:45.730 [SQSMessageConsumer-Thread-1] "Action=TagQueue&Version=2012-11
16:09:45.949 [SQSMessageConsumer-Thread-1] "Action=DeleteMessage&Version=20
16:09:46.155 [SQSMessageConsumer-Thread-1] "Action=ReceiveMessage&Version=2
16:09:46.393 [SQSMessageConsumer-Thread-1] "Action=ListQueueTags&Version=20
16:09:46.607 [SQSMessageConsumer-Thread-1] "Action=ListQueueTags&Version=20
16:09:46.818 [SQSMessageConsumer-Thread-1] "Action=SendMessage&Version=2012
16:09:47.026 [SQSMessageConsumer-Thread-1] "Action=DeleteMessage&Version=20
16:09:47.056 [SQSMessageConsumer-Thread-3] "Action=DeleteMessage&Version=20
16:09:47.234 [SQSMessageConsumer-Thread-1] "Action=ReceiveMessage&Version=2
16:09:47.263 [SQSMessageConsumer-Thread-3] "Action=ReceiveMessage&Version=2
16:09:47.679 [SQSMessageConsumer-Thread-1] "Action=ListQueueTags&Version=20
16:09:47.680 [ForkJoinPool.commonPool-worker-1] "Action=ListQueueTags&Versi
16:09:47.890 [SQSMessageConsumer-Thread-1] "Action=ListQueueTags&Version=20
16:09:47.900 [ForkJoinPool.commonPool-worker-1] "Action=ListQueueTags&Versi

The start of the runaway process seems to correspond with the first occurrence of AmazonSQSIdleQueueDeletingClient - Checking all queues begining with prefix appearing in the logs. (I guess the random initialDelay in IdleQueueSweeper causes the delay in the first occurrence of the problem.)

I wonder if #28 is not related (is not the cause)?

We are currently testing temporary queues for request/response messaging.
Functionality is working fine.

But created virtual queues are not getting deleted even after 5 minutes default time period.
And I am doing this testing with latest version of amazon-sqs-java-temporary-queues-client i.e. 1.0.1

Producer Code

`AmazonSQS sqs = AmazonSQSClientBuilder.standard()
.withRegion(Regions.US_EAST_1)
.build();

AmazonSQSRequester requester = AmazonSQSRequesterClientBuilder.standard()
.withAmazonSQS(sqs)
.withIdleQueueSweepingPeriod(60, TimeUnit.SECONDS)
.build();
String requestQueueUrl = sqs.createQueue("Simple").getQueueUrl();
SQSRequestor sqsRequestor = new SQSRequestor(requester, requestQueueUrl);
SendMessageRequest request = new SendMessageRequest()
.withQueueUrl(requestQueueUrl)
.withMessageBody(msg);
Message reply = sqs.sendMessageAndGetResponse(request, MAX_TIMEOUT, TimeUnit.SECONDS);`

Consumer Code

`AmazonSQS sqs = AmazonSQSClientBuilder.defaultClient();
String requestQueueUrl = sqs.createQueue("Simple").getQueueUrl();
AmazonSQSResponder responder = AmazonSQSResponderClientBuilder.standard().withAmazonSQS(sqs).build();
SQSResponder server = new SQSResponder(responder, requestQueueUrl);
//code in accept method

MessageContent reply = new MessageContent(response.getBody(), response.getMessageAttributes());
responseClient.sendResponseMessage(MessageContent.fromMessage(message), reply);`

Hello, I want to ask a scenario about failover.
Assume that client sent a message with sendMessageAndGetResponse.
Then it crashed after 3rd action, the message processed by the server, and the response written back to the temporary queue.
Is there any way to read that response message?

        1  - creates a temporary queue
        2  - attaches its URL as an attribute on the message
        3  - sends the message
        4  - receives the response from the temporary queue
        5  - deletes the temporary queue
        6  - returns the response

could you update Log as
private static final Log LOG = LogFactory.getLog(IdleQueueSweeper.class);

We are currently testing temporary queues with our apis for request/response messaging. After running the api the messages are passed and response is received as expected. Can see the queues on sqs console too.

After 5 min the idleSweepPeriods thread runs to check for idle queues and multiple virtual queue are created and deleted can seem them written to application logs continuously.

 INFO [SQSMessageConsumer-Thread-9] (AmazonSQSVirtualQueuesClient.java:190) – Deleting Virtual Queue is 6 and Queue Name is https://sqs.us-west-1.amazonaws.com/385242147137/markk-api-queue85eb0b0b-854c-456c-baae-0dc69dfc8a5d-0#markk-api-queue320a6d11-1d0a-48b5-ab69-29588c722e36
 INFO [SQSMessageConsumer-Thread-2] (AmazonSQSVirtualQueuesClient.java:161) – Total Virtual Queue Created is 7 and Queue Name is markk-api-queueb47c3650-5980-4bdf-8e99-8b8336303286
 INFO [SQSMessageConsumer-Thread-9] (AmazonSQSVirtualQueuesClient.java:190) – Deleting Virtual Queue is 6 and Queue Name is https://sqs.us-west-1.amazonaws.com/385242147137/markk-api-queue85eb0b0b-854c-456c-baae-0dc69dfc8a5d-0#markk-api-queueb47c3650-5980-4bdf-8e99-8b8336303286
 INFO [SQSMessageConsumer-Thread-2] (IdleQueueSweeper.java:49) – Checking all queues begining with prefix markk-api-queue for idleness
 INFO [SQSMessageConsumer-Thread-2] (AmazonSQSVirtualQueuesClient.java:161) – Total Virtual Queue Created is 7 and Queue Name is markk-api-queueaa9a124d-0cec-4c9e-97c0-39d9993351ee
 INFO [SQSMessageConsumer-Thread-2] (AmazonSQSVirtualQueuesClient.java:161) – Total Virtual Queue Created is 8 and Queue Name is markk-api-queue628ecf70-03b2-41c8-ad56-c6838973cf21
 INFO [SQSMessageConsumer-Thread-2] (IdleQueueSweeper.java:49) – Checking all queues begining with prefix markk-api-queue for idleness

Comparing milliseconds to seconds in https://github.com/awslabs/amazon-sqs-java-temporary-queues-client/blob/master/src/main/java/com/amazonaws/services/sqs/AmazonSQSIdleQueueDeletingClient.java#L300

We are observing excessive orphaned message warnings like below. I have grabbed these logs when there are no application activities happening, i.e. application is idle not servicing any request/response SQS related traffic during this period.
"com.amazonaws.services.sqs.AmazonSQSVirtualQueuesClient","message":"Orphaned message sent to __RequesterClientQueues__f
4ec3578-16ea-4206-9675-f11ccd910732: 0d580fc2-e160-4033-b593-5dc3121a7871","endOfBatch":true,"parameterCount":0,"formattedMessage":"Orphaned message sent to __RequesterClientQueues__f4ec3578-16ea-4206-9675-f11ccd910
732: 0d580fc2-e160-4033-b593-5dc3121a7871","loggerFqcn":"org.apache.commons.logging.impl.SLF4JLocationAwareLog","threadPriority":5,"threadId":128,"}

In one hour idle period, 905 orphaned msgs were observed on one deployment system, and 1021 orphaned msgs on second deployment system. These behavior is consistent observed since our production launch couple of weeks back.

In certain cases, we have observed these messages lead to SQS request throttling errors. Our system is ramping up traffic in production and would like to understand root cause of this errors as well as any potential issues with request throttling.

When using the requester client our Spring Boot application logs are contantly being filled with:

2020-10-28 13:10:58.300 INFO 8 --- [nsumer-Thread-1] c.a.s.s.AmazonSQSIdleQueueDeletingClient : Checking all queues begining with prefix __RequesterClientQueues__ for idleness
2020-10-28 13:10:58.335 ERROR 8 --- [nsumer-Thread-1] c.a.services.sqs.util.SQSQueueUtils : Unexpected exception
java.lang.RuntimeException: Exception encountered while processing message with ID 9cd43da3-34d6-4918-af84-1693dbb44462
at com.amazonaws.services.sqs.util.SQSMessageConsumer.handleMessage(SQSMessageConsumer.java:152) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) ~[na:na]
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159) ~[na:na]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[na:na]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[na:na]
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661) ~[na:na]
at com.amazonaws.services.sqs.util.SQSMessageConsumer.poll(SQSMessageConsumer.java:120) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: com.amazonaws.services.sqs.model.AmazonSQSException: Tag values must be between 0 and 256 characters in length. (Service: AmazonSQS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 26a1d477-5156-5f2e-b6df-8c6421749e63)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) ~[aws-java-sdk-core-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2214) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2181) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2170) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.executeTagQueue(AmazonSQSClient.java:2059) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSClient.tagQueue(AmazonSQSClient.java:2031) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.util.AbstractAmazonSQSClientWrapper.tagQueue(AbstractAmazonSQSClientWrapper.java:195) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.AmazonSQSVirtualQueuesClient.lambda$tagQueue$13(AmazonSQSVirtualQueuesClient.java:245) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at java.base/java.util.Optional.orElseGet(Optional.java:369) ~[na:na]
at com.amazonaws.services.sqs.AmazonSQSVirtualQueuesClient.tagQueue(AmazonSQSVirtualQueuesClient.java:245) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.util.AbstractAmazonSQSClientWrapper.tagQueue(AbstractAmazonSQSClientWrapper.java:195) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.AbstractAmazonSQS.tagQueue(AbstractAmazonSQS.java:223) ~[aws-java-sdk-sqs-1.11.601.jar!/:na]
at com.amazonaws.services.sqs.SQSExecutorService$Metadata.saveToTag(SQSExecutorService.java:159) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.SQSExecutorService$SQSFutureTask.done(SQSExecutorService.java:379) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at java.base/java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:381) ~[na:na]
at java.base/java.util.concurrent.FutureTask.setException(FutureTask.java:250) ~[na:na]
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:na]
at com.amazonaws.services.sqs.SQSScheduledExecutorService$ScheduledSQSFutureTask.access$201(SQSScheduledExecutorService.java:33) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.SQSScheduledExecutorService$ScheduledSQSFutureTask.run(SQSScheduledExecutorService.java:147) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.SQSExecutorService.accept(SQSExecutorService.java:196) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.util.SQSMessageConsumer.accept(SQSMessageConsumer.java:166) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]
at com.amazonaws.services.sqs.util.SQSMessageConsumer.handleMessage(SQSMessageConsumer.java:145) ~[amazon-sqs-java-temporary-queues-client-1.2.0.jar!/:na]

Hello,
I'm investigating the SQS temporary queues feature.

I've written a small Kotlin script to test the feature, and I come up with strange results.
The script and details can be found here

I'm getting very strange results, and I see that messages are often not being processed/treated/sent at all and eventually resulting in a timeout exception(I assume).
This is very critical, and I'm not sure if I'm using the library in the wrong way.

In general, I'm going to use this feature for the following use case:

1. On demand, create a host queue and provision a server that will consume messages for that queue. Each operation should create a new queue and server.
2. Make HTTP calls and pass messages through that channel (temporary queues) and wait till the response is back, and then the HTTP request can return the response.
3. Eventually, I should have a "client" server that contains the sqsRequester instance, and each user will use it to make calls to the right host queue based on its context, and many servers that will consume the messages, each server with a sqsResponder that is configured to listen to a specific host queue.

Please assist me here ❤️

Hi. My use case is: I'd like to implement a request-response system via SQS, and have a single responder handling requests from multiple requesters (without having to create a new SQS queue/queues for every requester, every time I add one). Is this something that is envisioned (or already possible) for amazon-sqs-java-temporary-queues-client?

Thanks

Hi

SQSMessageConsumer has a tight loop in poll(), summarised as:

for (;;) {
   try {
      // receive a message
   } catch (Exception e) {
      exceptionHandler.accept(e);
   }
}

When an exception occurs (for example, kill network connectivity for a java.net.UnknownHostException: sqs.eu-west-1.amazonaws.com), this spins away and generates a huge stack trace every few milliseconds, and consumes all CPU resources very quickly.

Please consider sleeping a second upon exception.

Thanks

Hello.

We have implemented this and during testing we are seeing some errors in the logs.

Caused by: com.amazonaws.services.sqs.model.AmazonSQSException: Request is throttled. (Service: AmazonSQS; Status Code: 403; Error Code: RequestThrottled; Request ID: c8096cbe-1be5-5b2e-8a18-3a9c5fe245d8) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2235) at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2202) at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2191) at com.amazonaws.services.sqs.AmazonSQSClient.executeTagQueue(AmazonSQSClient.java:2043) at com.amazonaws.services.sqs.AmazonSQSClient.tagQueue(AmazonSQSClient.java:2015) at com.amazonaws.services.sqs.util.AbstractAmazonSQSClientWrapper.tagQueue(AbstractAmazonSQSClientWrapper.java:195) at com.amazonaws.services.sqs.AmazonSQSVirtualQueuesClient.lambda$tagQueue$13(AmazonSQSVirtualQueuesClient.java:235)

We suspect this is related to the service limit on our account (which is initially set to 5 TPS).
Do you have any recommendation as to which is a good limit to request when submitting a request for service limit increase ?

Please upgrade the jackson dependencies.

Dependencies (13)
Dependency: MAVEN - commons-codec:commons-codec:1.11:jar
Dependency: MAVEN - commons-logging:commons-logging:1.1.3:jar
Dependency: MAVEN - software.amazon.ion:ion-java:1.0.2:jar
Dependency: MAVEN - com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.6.7:jar
Dependency: MAVEN - joda-time:joda-time:2.8.1:jar
Dependency: MAVEN - com.fasterxml.jackson.core:jackson-annotations:2.6.0:jar
Dependency: MAVEN - com.fasterxml.jackson.core:jackson-core:2.6.7:jar
Dependency: MAVEN - com.fasterxml.jackson.core:jackson-databind:2.6.7.3:jar
RejectReasons (11)
RejectReason: 450d442b-54b1-40e7-8ad4-2d5fa52ebe69
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-467016
CVSS Score v3: 9.8
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-467016
RejectReason: 752677dc-d7c4-45f9-b451-fb6bf16a61d5
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-467015
CVSS Score v3: 9.8
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-467015
RejectReason: 55c7dec2-4e8b-4fbf-bce4-7c7f16382f4b
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-467014
CVSS Score v3: 9.8
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-467014
RejectReason: b585c1c1-c19b-4455-8315-5ee412d54a14
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-174736
CVSS Score v3: 9.8
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-174736
RejectReason: 55db71fe-6d83-43e2-94e3-bb08a6b632c7
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72884
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72884
RejectReason: 18b261c7-73ed-460c-894f-84027df9d762
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72883
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72883
RejectReason: 95f0c3a1-b16d-4da5-bbca-7cad66bd5d12
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72882
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72882
RejectReason: 0a03b1fd-17e8-441b-a7d3-6b0e1a9d49ba
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72447
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72447
RejectReason: 3cbb405a-0b9b-4efb-8132-a9eeff58d00b
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72446
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72446
RejectReason: 92f0f2cf-fa32-4f55-b3df-a09ee00db392
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-72445
CVSS Score v3: 10
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-72445
RejectReason: 035709b2-d8c3-4a2a-b01f-d93e2c5077e0
Type: VULNERABILITY
Name: SNYK-JAVA-COMFASTERXMLJACKSONCORE-32111
CVSS Score v3: 9.8
Severity: critical
Description Link: https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-32111

Is it correct that a virtual queue cannot subscribe to SNS topic?

This looks like a very interesting project, one that would replace some custom code we have where we create actual SQS queues for each requester (so as to send the responses to the correct requester).

Unfortunately, the projects where we need this are all in Python, and we don't want to add dependencies to py4j or the like.

Your documentation and the introductory post give some clues about the overall mechanism:

when sending a message, the virtual queue name is attached as an attribute to the message;

Ok, I can understand that, and how the receiver would then use the same attribute when sending the response. But when receiving a message, what is the mechanism actually used (on the real SQS queue) to filter messages for the requester?

BTW, my assumption is that we have distributed requesters; is that correct?

It would be good to provide some explanation about the underlying mechanism that is used.