Comments (5)
Hi @klefevre, thank you for reporting this. We've been able to reproduce the issue on our end. Will look into this further.
from aws-sdk-rust.
FYI, while we've reproduced it, it does not seem to be consistently reproducible. For instance, when I run the main function in aws-sdk-rust-concurrency-issue
4 times, with const CONCURRENCY_LIMIT: usize = 200
and const RANGE: Range<u32> = 1..2000
I get
1st run (success)
➜ aws-sdk-rust-concurrency-issue git:(main) ✗ cargo r
Finished dev [unoptimized + debuginfo] target(s) in 3.40s
Running `target/debug/aws-sdk-rust-concurrent-issue`
2nd run (connection timeout)
➜ aws-sdk-rust-concurrency-issue git:(main) ✗ cargo r
Finished dev [unoptimized + debuginfo] target(s) in 0.10s
Running `target/debug/aws-sdk-rust-concurrent-issue`
Error: Failed to collect objects
Caused by:
0: Failed to fetch object 1
1: dispatch failure
2: timeout
3: error trying to connect: HTTP connect timeout occurred after 3.1s
4: HTTP connect timeout occurred after 3.1s
5: timed out
3rd run (success)
➜ aws-sdk-rust-concurrency-issue git:(main) ✗ cargo r
Finished dev [unoptimized + debuginfo] target(s) in 0.10s
Running `target/debug/aws-sdk-rust-concurrent-issue`
4th run (dns error)
➜ aws-sdk-rust-concurrency-issue git:(main) ✗ cargo r
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Running `target/debug/aws-sdk-rust-concurrent-issue`
Error: Failed to collect objects
Caused by:
0: Failed to fetch object 281
1: dispatch failure
2: io error
3: error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known
This seems to indicate that we're pushing the use of underlying resources to the limit where it may or may not behave reliably. Could you explain what makes you use 200 (or more) concurrent GetObject
?
from aws-sdk-rust.
Could you explain what makes you use 200 (or more) concurrent GetObject
Chiming in as well since this helps my current use case. We're trying to migrate millions of objects as fast as we can from a foreign S3 bucket (on a third party's AWS account), some of those objects being bigger than 800GB, so we'll be leveraging this blogpost to max out the amount of concurrent connections too, not only the amount of concurrent S3 Batch Operations objects.
OTOH, it seems that in order to implement the aforementioned AWS blogpost in Rust we'll be limited by this other issue first, though: #968, we'd need to have those parameters configurable:
max_concurrency: 940, max_retries: 100, max_pool_connections: 940 and multipart_chunksize: 16777216.
from aws-sdk-rust.
I found a resolution to the problem. I was actually expecting the SDK to return only two kinds of errors in this scenario: network issues with S3 itself or a failure to open a file descriptor due to a limit reached on my hardware.
While writing this response, I checked the maximum number of file descriptors I could open with the command ulimit -n
, and it turns out to be 256 by default (at least on macOS). By removing this limit i.e. ulimit -n unlimited
, I no longer encounter errors. 🙌
I'm closing this issue because the SDK behaves correctly. However it would be nice to add the root cause of this dns error for better clarity and troubleshooting in the future.
from aws-sdk-rust.
Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.
from aws-sdk-rust.
Related Issues (20)
- AWS SDK for Rust doesn't take into account kube2iam authentication HOT 2
- Hyper dns error when using docker HOT 4
- Incorrect s3 metadata in Presigned URLs HOT 3
- Allow presigning to hoist more values into the URI
- Disable StalledStreamProtection for S3 CopyObject HOT 14
- ListPartsPaginator for s3 list_parts operation goes into infinite loop HOT 3
- very big stack when optimisation disabled HOT 2
- SQS `ReceiveMessageFluentBuilder` `fn attribute_names` deprecation HOT 2
- Unable to perform multipart upload after upgrade: current throughput: 0 B/s is below minimum: 1 B/s HOT 6
- Upgrade to `aws-sdk-sqs="1.23.0"` causing timeouts HOT 10
- misspellings of "supersede"
- SSM GetParametersInputBuilder.names Doesn't Split Parameters HOT 3
- add new option `overwrite_by_pkeys` for DynamoDB `batch_write_item` HOT 1
- `smithy connection was poisoned` with latest aws_smithy_runtime HOT 6
- Random spikes in dynamodb update latency HOT 6
- Add examples for Waiters Implementation HOT 1
- S3 Transfer Manager High Level Library
- Link to amazon envvars configuration HOT 5
- S3 `complete_multipart_upload` are not retried automatically on internal error HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-sdk-rust.