Code Monkey home page Code Monkey logo

rusoto's Introduction

Rusoto

api-docs-badge crates-io license-badge dependency-status-badge

Rusoto is an AWS SDK for Rust


You may be looking for:

Maintenance status

⚠️ Rusoto is in maintenance mode. ⚠️

The current maintainers only have the bandwidth to review dependency bumps and obvious bugfixes. Our bandwidth for reviewing new features is extremely limited.

While you are welcome to submit PRs that implement new features or refactor existing code, they are unlikely to be merged unless we can find more active maintainers.

Please see Meta: future of Rusoto (#1651) for details.

Installation

Rusoto is available on crates.io. To use Rusoto in your Rust program built with Cargo, add it as a dependency and rusoto_$SERVICENAME for any supported AWS service you want to use.

For example, to include only S3 and SQS:

[dependencies]
rusoto_core = "0.48.0"
rusoto_sqs = "0.48.0"
rusoto_s3 = "0.48.0"

Migration notes

Breaking changes and migration details are documented at https://rusoto.org/migrations.html.

Note that from v0.43.0 onward, Rusoto uses Rust's std::future::Future, and the Tokio 0.2 ecosystem. From v0.46.0 onward, Rusoto uses the Tokio 1.0 ecosystem.

Usage

Rusoto has a crate for each AWS service, containing Rust types for that service's API. A full list of these services can be found here. All other public types are reexported to the crate root. Consult the rustdoc documentation for full details by running cargo doc or visiting the online documentation for the latest crates.io release.

A simple example of using Rusoto's DynamoDB API to list the names of all tables in a database:

use rusoto_core::Region;
use rusoto_dynamodb::{DynamoDb, DynamoDbClient, ListTablesInput};

#[tokio::main]
async fn main() {
    let client = DynamoDbClient::new(Region::UsEast1);
    let list_tables_input: ListTablesInput = Default::default();

    match client.list_tables(list_tables_input).await {
        Ok(output) => match output.table_names {
            Some(table_name_list) => {
                println!("Tables in database:");

                for table_name in table_name_list {
                    println!("{}", table_name);
                }
            }
            None => println!("No tables in database!"),
        },
        Err(error) => {
            println!("Error: {:?}", error);
        }
    }
}

Credentials

For more information on Rusoto's use of AWS credentials such as priority and refreshing, see AWS Credentials.

Semantic versioning

Rusoto complies with semantic versioning 2.0.0. Until reaching 1.0.0 the API is to be considered unstable. See Cargo.toml or rusoto on crates.io for current version.

Releases

Information on release schedules and procedures are in RELEASING.

Contributing

Discussions take place on the Rusoto Discord channel.

See CONTRIBUTING for more information.

Supported OSs, Rust versions and non-AWS projects

Linux, macOS and Windows are supported and tested via GitHub actions.

Rust stable, beta and nightly are supported.

Rusoto's primary aim is to be used with AWS. Other projects that provide AWS-like APIs, such as Ceph, Minio, Yandex Object Storage, etc... are not a focus at this time. PRs to fix issues with Rusoto and AWS-like APIs are welcome but generally won't be created by Rusoto maintainers.

License

Rusoto is distributed under the terms of the MIT license.

See LICENSE for details.

rusoto's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rusoto's Issues

Add IAM role ability

Use credentials from IAM role.

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/

returns the IAM role. In my case: jenkins.

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/jenkins

returns

{
  "Code" : "Success",
  "LastUpdated" : "2015-08-04T00:09:23Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "AAAAAA",
  "SecretAccessKey" : "AAAAA",
  "Token" : "AAAAA",
  "Expiration" : "2015-08-04T06:32:37Z"
}

A request to an IAM role that doesn't exist will return HTTP 404.

S3 file upload: check use of .clone()

We're cloning the data passed in to the S3 client. This is suboptimal for performance. Figure out the lifetimes to use the data passed in rather than cloning it.

Meta: modify info about sample main.rs program

Update the README and maybe branch out to a new file that goes over what main.rs does.

Maybe a sample or two of how to use S3 and SQS in the README and better documentation on the main.rs code sample.

This might tie into the better documentation story we have. There's an issue for that, right?

Meta: Investigate rustfmt

Automatic formatting would be nice. rustfmt requires nightly Rust builds and may not be ready yet. Investigate to see if we can get anything useful out of it and how.

Credentials: ensure refreshing works as expected

Right now our S3Client and SQSClient classes take a AWSCredentials reference, which contains the access key and secret key. When those expire the client doesn't know.

We could refactor so the *Client classes take the provider chain and handle calling get_credentials() themselves. This would hide expiration from Rusoto users.

Example use case: on an EC2 instance with IAM instance profile for creds, a thread polls an SQS queue periodically. After ~5 minutes the credentials will expire and it will have to get new ones.

Credentials: use aggressive IAM timeout

If the credentials chain fails on finding environment or file based credentials, say from providing a bad credentials profile name, we're calling the IAM metadata service to find creds. If we're running on a dev laptop it will sit until it times out which is probably 60 seconds.

Let's drop that timeout to say 15 seconds.

Meta: Generate better error handling code from botocore definitions

Botocore specifies the errors that can be returned from requests in a lot of places. We should investigate a way to leverage that with botocore_parser in our generated code where we can, instead of returning a generic AWSError everywhere something goes wrong.

e.g., from the JSON definition of the DynamoDB create_table method:

 "errors":[
        {
          "shape":"ResourceInUseException",
          "exception":true,
          "documentation":"<p>The operation conflicts with the resource's availability. For example, you attempted to recreate an existing table, or tried to delete a table c\
urrently in the <code>CREATING</code> state.</p>"
        },
        {
          "shape":"LimitExceededException",
          "exception":true,
          "documentation":"<p>The number of concurrent table requests (cumulative number of tables in the <code>CREATING</code>, <code>DELETING</code> or <code>UPDATING</code\
> state) exceeds the maximum allowed of 10.</p> <p>Also, for tables with secondary indexes, only one of those tables can be in the <code>CREATING</code> state at any point in\
 time. Do not attempt to create more than one such table simultaneously.</p> <p>The total limit of tables in the <code>ACTIVE</code> state is 250.</p>"
        },
        {
          "shape":"InternalServerError",
          "exception":true,
          "fault":true,
          "documentation":"<p>An error occurred on the server side.</p>"
        }
      ],

S3: add md5 hash to object uploads

We're not hashing the payload in the S3 object requests. We are doing it for the overall request but I think we need to supply the content-md5 header.

Might need to go in the request or signature file instead of S3.

Meta: rename repo to rusoto

Rename, or maybe copy this repo to leave this intact and link to new one.

Also requires update to our Cargo file so the source link is in the right spot.

Split signature and request responsibilities.

Right now signatures.rs handles both creating the signature for the request and executing it. Let's split those.

This can help #63 by setting follow_redirects to false and resending the request to the temporary endpoint.

Credentials: support credential file roles

Right now the file based credential provider just grabs the first set of credentials in the file. Add ability to use profiles. I think the env var AWS_PROFILE is one method of specifying profiles. Check other SDKs to see how they are supporting this.

Code generation doesn't handle commands that don't have inputs

Python code:

# generate rust code to sign and execute an HTTP request for a botocore operation
def request_method(operation):
    http = operation['http']
    if not ('input' in operation):
        return

S3 List buckets JSON definition from botocore:

"ListBuckets":{
      "name":"ListBuckets",
      "http":{
        "method":"GET",
        "requestUri":"/"
      },
      "output":{"shape":"ListBucketsOutput"},
      "documentation":"Returns a list of all buckets owned by the authenticated sender of the request.",
      "documentationUrl":"http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTServiceGET.html",
      "alias":"GetService"
    },

No list_buckets function is generated.

Fix code gen to handle this case.

CI: Drone not updating Rust Docker image

Our Drone instance isn't picking up the latest Rust Docker image, despite Drone docs saying it should.

We need to get this working as it's blocking use of the latest hyper library, which is in turn blocking other issues like #55.

Without this we're stuck on Rust 1.1.0 and we should really be compiling against the latest stable release.

xmlutils are currently un-unit-testable

Xmlutils uses a type that wraps the Hyper library's Response type which makes it impossible to test without an external web service that stands in for AWS. Or just calling AWS.

To facilitate unit testing and TDD, this should be refactored.

Code generation will likely have to be modified to use the new type instead of XmlStack.

Check use of credentials to ensure rereshing

Right now we only fetch credentials once in our sample code. This will break with IAM instance profile credentials since they will expire after a set amount of time.

Update sample code to use the get_credentials function whenever credentials are needed.

Probably good to come up with a simple service that can be used as integration testing to verify and as an example for users of rusoto.

S3 actions right after bucket creation: handle 307 redirects

As seen in #28: if you create a bucket anywhere but us-east-1 and immediately try to upload objects to it, you'll get a 307 temporary redirect. Hyper appears to follow that but the message body does not get re-sent, hence the errors I saw where AWS calculated a body payload of empty string.

This ticket is for reproducing this behavior locally and verifying it's a bug in Hyper, then giving them an easy repro in a ticket.

Update README example of SQS client

The README has an outdated sample for how the SQS client works. Let's change it to this, almost directly from the main.rs file:

    let provider = DefaultAWSCredentialsProviderChain::new();
    let region = Region::UsEast1;

    let mut sqs = SQSHelper::new(provider, &region);

    let response = try!(sqs.list_queues());
    for q in response.queue_urls {
        println!("Existing queue url: {}", q);
    }

Meta: come up with release/tag process

Need a document to describe how we do releases, tagging commits, etc...

Should cover:

  • git tags
  • publishing to crates.io
  • guideline for bumping version numbers

Add S3 capability

Add functionality for S3. There's a branch for this already (https://github.com/DualSpark/rust-aws/tree/feature-s3) but after #17 it may be best to dump it and start over. This is due to refactoring of code generation and ability to do TDD for generated code instead of doing integration testing with AWS.

(Edit for book keeping on progress)
The AWS CLI offers these functions (http://docs.aws.amazon.com/cli/latest/reference/s3api/index.html):

abort-multipart-upload ✔️
complete-multipart-upload ✔️
copy-object
create-bucket ✔️
create-multipart-upload ✔️
delete-bucket ✔️
delete-bucket-cors
delete-bucket-lifecycle
delete-bucket-policy
delete-bucket-replication
delete-bucket-tagging
delete-bucket-website
delete-object ✔️
delete-objects
get-bucket-acl
get-bucket-cors
get-bucket-lifecycle
get-bucket-location
get-bucket-logging
get-bucket-notification
get-bucket-notification-configuration
get-bucket-policy
get-bucket-replication
get-bucket-request-payment
get-bucket-tagging
get-bucket-versioning
get-bucket-website
get-object ✔️
get-object-acl
get-object-torrent
head-bucket
head-object
list-buckets ✔️
list-multipart-uploads ✔️
list-object-versions
list-objects
list-parts ✔️
put-bucket-acl
put-bucket-cors
put-bucket-lifecycle
put-bucket-logging
put-bucket-notification
put-bucket-notification-configuration
put-bucket-policy
put-bucket-replication
put-bucket-request-payment
put-bucket-tagging
put-bucket-versioning
put-bucket-website
put-object ✔️
put-object-acl
restore-object
upload-part ✔️
upload-part-copy
wait

Only compile tests when testing

We need to add #[cfg(test)] to our tests.

Might need another issue to ensure we're doing idiomatic Rust testing with how tests are laid out and their module names.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.