tedder / requests-aws4auth Goto Github PK

Amazon Web Services version 4 authentication for the Python Requests module

License: MIT License

Python 100.00%

requests-aws4auth's Issues

Version bump to 0.7?

To install off pypi I think we need a version bump or update for 0.6 to get #2 out in the wild. How do you prefer to do that?

Use Cross-account Roles?

is there a way to specify the arn of the role with requests-aws4auth?

context: I'm trying to implement cross-account access like this:
https://docs.aws.amazon.com/athena/latest/ug/cross-account-permissions.html

ps: happy to pick this if this needs to be added

Thanks

AWS authentication failed using HTTPS request

I'm deploying CSR 1000v on an EC2 instance in AWS.

This is my python code for authentication in order to use RESTCONF which is already enabled in the router.

import requests
from requests_aws4auth import AWS4Auth
import pprint
import urllib3

 
def get_json(interface):
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

    authaws = AWS4Auth('AWS_ACCESS_KEY',
                       'AWS_SECRET_ACCESS_KEY',
                       'us-west-2',
                       'awis')
 
    source = 'https://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com/restconf/data/'
    module = 'ietf-interfaces:'
    container = 'interfaces'
    leaf = '/interface=' + interface
    options = ''
 
    url = source + module + container + leaf + options
    headers = {'Content-type': 'application/yang-data+json', 'Accept': 'application/yang-data+json'}
 
    r = requests.get(url, auth=authaws, headers=headers, verify=False)
 
    return r.json()
 
if __name__ == '__main__':
 
    interface = 'GigabitEthernet1'
 
    pprint.pprint(get_json(interface))

Here what I got after execution.

server@zsz:~/shared_files$ python get_one_interface.py 
{u'errors': {u'error': [{u'error-tag': u'access-denied',
                         u'error-type': u'protocol'}]}}

Obviously, the authentication cannot be done.
For aws_access_key and aws_secret_access_key, I got it from IAM console. I even generated new ones, but still does not work.

AWS error response when request date does not match signing key date

Description

Reported by @ipartola in pull request #3.

When the date of a request, as given in either the Date or the X-Amz-Date header, is different from the date in the AWS4Auth object's signing key scope, then an invalid signature is generated and AWS sends an authentication error in response to the request.

This issue affects request-aws4auth versions 0.7 and earlier.

As descibed by @ipartola:

"The bug is that the timestamp for the signing key is generated when the object is instantiated. Once we cross a date boundary it seems AWS requires that the signing key's scope date match the x-amz-date header. The solution I propose is to continually set the current date for the signing key to match the x-amz-date header."

Discussion

The AWS documentation for date handling in signing key scopes is here: http://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html

It could be argued that this isn't a bug in requests-aws4auth, since it should be up to the calling code to ensure that the scope of the signing key is kept up to date by generating a new AWS4Auth instance. But it's obvious that this is a common situation and that it would be a very useful feature if requests-aws4auth could handle this automatically.

Allowing the key to automatically regenerate raises problems for sharing an AWS4Auth instance between multiple threads, since if the key can be regenerated automatically at any time then it could be modified by another thread while it is being used to sign a request.

It also means that the AWS secret key will need to be stored in the signing key, so that the key can be regenerated without having to escape back out to the code generating the request.

Proposed Solution

As suggested by @ipartola, the key needs to be regenerated with a date that matches the date in the request.

As this is likely to be a common situation, and a bit of a WTF moment when new users hit it, it is proposed that automatic key regeneration is made the default behaviour for new instances of AWS4Auth. However, the behaviour needs to be made configurable, for situations where people want to share instances between threads, or don't want the key stored in the instance, or don't want automatic regeneration for other reasons. Retaining a way of duplicating the current behaviour (i.e. signing the request with a bad signature) is also desirable for backward compatibility if needed.

The proposal is to modify the AWS4Auth class to check a request's X-Amz-Date or Date headers against the scope date, and if it is different then automatically regenerate the signing key with a scope that uses the new date. This means when new instances of AWS4Auth are created, the AWS secret key will need to be supplied.

A new method will be added to AWS4Auth: `regenerate_signing_key()``. Calling this will trigger an immediate regeneration of the instance's signing key. Parameters can be supplied to this method to set new values for the key scope parameters: region, service and date. This method will be used when a automatic key regeneration is needed, but can also be called by other code if a manual regeneration is needed.

Two new subclasses of AWS4Auth will be added: StrictAWS4Auth and PassiveAWS4Auth. StrictAWS4Auth will not regenerate the signing key when a date mismatch is encountered, it will instead raise a new exception, DateMismatchError. This will allow the calling code to handle the situation itself if desired.

PassiveAWS4Auth will do no date checking, and will sign the request and send it even if the request date does not match the scope date. This mimics the current behaviour of v0.7 AWS4Auth.

Further, a new parameter will be added to AWS4SigningKey: store_secret_key. If this is set to True, which will be the default, the secret key is stored in the instance, not thrown away as it is in v0.7. By creating instances of AWS4Auth or its subclasses using AWS4SigningKey instances created with store_secret_key set to False, you can control whether the secret key is stored in an AWS4Auth instance or not.

Examples

Regular usage, usual case with no date change, X-Amz-Date header is automatically added to the request by the AWS4Auth signing process, using today's date, since Requests doesn't automatically add a date header. This is the same usage and behaviour as current 0.7 version when not crossing a date boundary:

>>> auth = AWS4Auth('access_id', 'secret_key', 'us-east-1', 's3')
>>> auth.date
'20151222'
>>> response = requests.get('http://s3.amazonaws.com', auth=auth)
>>> response.text
<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01">
  <Owner>
    <ID>bcaf1ffd86f461ca5fb16fd081034f</ID>
    <DisplayName>webfile</DisplayName>
>>> response.requests.headers['X-Amz-Date']
'20151222T212208Z'

Next this shows regular usage, but crossing a date boundary and AWS4Auth automatically regenerating the signing key. This is the main use case for this fix. (In this case the X-Amz-Date header is again added automatically by the AWS4Auth authentication using the current date. It is added before the scope/request date check is made, and can be different to the scope date):

>>> auth = AWS4Auth('access_id', 'secret_key', 'us-east-1', 's3')
>>> auth.date
'20151222'
>>> id(auth.signing_key)
11162416
...wait until tomorrow...
>>> datetime.utcnow().strftime('%Y-%m-%d')
'2015-12-23'
>>> response = requests.get('http://s3.amazonaws.com', auth=auth)
>>> response.text
<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01">
  <Owner>
    <ID>bcaf1ffd86f461ca5fb16fd081034f</ID>
    <DisplayName>webfile</DisplayName>
>>> response.request.headers['X-Amz-Date']
'20151232T000934Z'
>>> auth.date
'20151223'
>>> id(auth.signing_key)
10947184

Next is an example with StrictAWS4Auth. With this subclass the key is not automatically regenerated, instead a DateMismatchError is raised when the request date does not match the signing key scope date:

>>> auth = StrictAWS4Auth('access_id', 'secret_key', 'us-east-1', 's3')
>>> auth.date
'20151222'
>>> id(auth.signing_key)
11162416
...wait until tomorrow...
>>> datetime.utcnow().strftime('%Y-%m-%d')
'2015-12-23'
>>> response = requests.get('http://s3.amazonaws.com', auth=auth)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  ...
requests_aws4auth.exceptions.DateMismatchError

The key can now be manually regenerated to match the new day's date:

>>> today = datetime.utcnow().strftime('%y%m%d')
>>> today
'20151223'
>>> auth.regenerate_signing_key(date=today)
>>> auth.date
'20151223'
>>> id(auth.signing_key)
10947184

Example again using StrictAWS4Auth, but also where the secret key is not stored in the auth instance:

>>> sig_key = AWS4SigningKey('secret_key', 'us-east-1', 's3', store_secret_key=False)
>>> auth = AWS4Auth('access_id', sig_key)
>>> auth.date
'20151222'
>>> id(auth.signing_key)
11162416
...wait until tomorrow...
>>> datetime.utcnow().strftime('%Y-%m-%d')
'2015-12-23'
>>> response = requests.get('http://s3.amazonaws.com', auth=auth)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  ...
requests_awsauth.exceptions.DateMismatchError
>>> today = datetime.utcnow().strftime('%y%m%d')
>>> auth.regenerate_signing_key(date=today)
requests_aws4auth.exceptions.NoSecretKeyError

Here the manual regeneration fails because there is no stored secret key. To regenerate, the secret key must be passed to regenerate_signing_key():

>>> auth.regenerate_signing_key(secret_key='secret_key', date=today)
>>> auth.date
'20151223'
>>> id(auth.signing_key)
10947184

The key is not stored in the new key either, because the new key uses the value of store_secret_key in the old key, which isFalse in this case.

Any feedback on these proposed changes gratefully received.

FYI: works with Tornado's HTTPRequest objects

Not an issue, just an FYI
requests-aws4auth seems to work fine with Tornado's HTTPRequest objects. I use it for lightweight AWS requests, like uploading files to S3. This saves a lot of boto3 or botocore dependencies and code and I can use the asynchronous AsyncHttpClient, leveraging Tornado's async/non-blocking/ioloop features.

The HTTPRequest object exposes a headers dict and a body property, which seems to be all AWS4Auth needs.

For example:

auth = requests_aws4auth.AWS4Auth(<key>, <secret>, "eu-west-1", "s3")
req = tornado.httpclient.HTTPRequest(<endpoint>, method="PUT", body=<body>)
signed_req = auth(req)

tornado.httpclient.AsyncHTTPClient().fetch(signed_req)

That is all, thanks ;)

Support for Appsync?

Any idea if appsync is supported? Not in the listed supported services.

Exception when using an empty "secret key" string

If you get an error like this, you're creating AWS4Auth() with an empty "secret key" string.

>>> requests_aws4auth.AWS4Auth('username','', 'eu-west-1', 's3')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py", line 246, in __init__
    self.regenerate_signing_key(secret_key=secret_key)
  File "/usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py", line 291, in regenerate_signing_key
    secret_key = secret_key or self.signing_key.secret_key
AttributeError: 'NoneType' object has no attribute 'secret_key'

It's not something real users would want to do, but found this in testing code that previously ran with V2 Signature awsauth.S3Auth. This patch works around the error:

diff -u /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py~ /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py
--- /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py~	2017-02-15 19:17:02.000000000 +0000
+++ /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py	2017-02-18 00:18:15.425646504 +0000
@@ -288,7 +288,7 @@
                                    self.signing_key.secret_key is None):
             raise NoSecretKeyError
 
-        secret_key = secret_key or self.signing_key.secret_key
+        secret_key = secret_key or (self.signing_key and self.signing_key.secret_key) or ''
         region = region or self.region
         service = service or self.service
         date = date or self.date

PEP440 warning

When installing another package that requires aws4auth the package the log output shows:

Searching for requests-aws4auth>=0.9.0
Reading https://pypi.python.org/simple/requests-aws4auth/
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.8)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.5)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.7)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.9)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.4)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'requests (aws4auth-0.6)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
Best match: requests-aws4auth 0.9
Downloading https://pypi.python.org/packages/72/97/ec440cf78418a62ed8351424dfb6590525e8f95cf634cb35e0075f8d3706/requests-aws4auth-0.9.tar.gz#md5=572b55f37149610c7de0a19eaac51599
Processing requests-aws4auth-0.9.tar.gz
Writing /tmp/easy_install-juBBPo/requests-aws4auth-0.9/setup.cfg
Running requests-aws4auth-0.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-juBBPo/requests-aws4auth-0.9/egg-dist-tmp-RKKO9w
zip_safe flag not set; analyzing archive contents...
requests_aws4auth.six: module references __path__
creating /usr/local/lib/python2.7/site-packages/requests_aws4auth-0.9-py2.7.egg
Extracting requests_aws4auth-0.9-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding requests-aws4auth 0.9 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/requests_aws4auth-0.9-py2.7.egg

Function amz_cano_querystring() does not work correctly.

I found problem about oder of canonicail querystring.
It is not sorted correctly

>>> from requests_aws4auth import AWS4Auth
>>> qs = 'key=&key-type=s3&format=json'
>>> AWS4Auth.amz_cano_querystring(qs)
'format=json&key-type=s3&key='

Expected behavior

>>> AWS4Auth.amz_cano_querystring(qs)
'format=json&key=&key-type=s3'

Querystring keys consist of format, key-type and key.
After sorted, querystring keys should be format, key, key-type.
(key must sorted before key-type).
I think you need to rewrite the sort method.

Incorrect request signature

There is a bug in sorting query string parameters in case when parameter keys include dashes

https://github.com/sam-washington/requests-aws4auth/blob/master/requests_aws4auth/aws4auth.py#L661

Here parameters are sorted after they urlencoded, which puts them in a wrong order.

For example:

sorted("id=1000000161418039&id-type=receipt".split("&"))
>>> ['id-type=receipt', 'id=1000000161418039']

If you sort them by name, as AWS describes here http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html, the order would be the following

sorted([tuple(pair.split("=")) for pair in "id=1000000161418039&id-type=receipt".split("&")], key=lambda x: x[0])
>>> [('id', '1000000161418039'), ('id-type', 'receipt')]

is it time to fork this? can anyone contact Sam Washington?

This is still a useful library, despite its age. There are pending PRs. @sam-washington doesn't have any activity in github, they have a keybase entry for github and aethris.net, which is a dead site and doesn't exist in the Wayback Machine. The domain is still registered, though.

Python has a process for doing all of this:

As soon as I post this I'll send the issue link to [email protected], which is the only email address I've found.

Incorrect signature when send request with spaces in query parameters

I am trying to and a request to the api gateway with query string parameters that have a value that includes spaces. However, I keep getting a 403 explaining that I have a signature that does not match. When I replace the spaces with '+' the request works fine. How do I get it to work with the spaces?

def get_auth():
    # uses aws creds from env to get auth header for requests to api gateway endpoints

    session = boto3.Session()
    credentials = session.get_credentials()
    region = os.getenv("AWS_REGION")
    awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, 'execute-api', session_token=credentials.token)
    return awsauth


response = requests.get('https://xxxxxxxxxx.execute-api.us-west-2.amazonaws.com/staging?group_name=test group one&[email protected], auth=get_auth())

README, LICENSE, and HISTORY installed directly in site-packages

Due to some lines in setup.py, a requests-aws4auth installation ends up writing some files with generic names into the top-level site-packages directory:

$ python3 -m venv e
$ e/bin/pip install requests-aws4auth
$ head e/lib/python*/site-packages/README.md
[![image](https://img.shields.io/pypi/v/requests-aws4auth.svg)](https://pypi.python.org/pypi/requests-aws4auth)
[![image](https://img.shields.io/pypi/l/requests-aws4auth.svg)](https://pypi.python.org/pypi/requests-aws4auth)

Amazon Web Services version 4 authentication for the Python [Requests](https://github.com/kennethreitz/requests) library.

Features
========

-   Requests authentication for all AWS services that support AWS auth v4
-   Independent signing key objects

Why the "x-amz-date" in the headers is not "X-Amz-Date" ?

I saw the code in the https://github.com/tedder/requests-aws4auth/blob/master/requests_aws4auth/aws4auth.py everywhere in the description we have it as "X-Amz-Date" whereas while forming the actual headers it is "x-amz-date". What is the expected to be sent in the Sig v4 for s3 ?

Issue with Signing AWS Requests for Paths Starting with Double Slash

I have encountered an issue while using the library to sign AWS request, specifically when the requests start with a double slash (e.g., when an object in S3 contains a forward slash '/'). After investigating the source code, it appears that the problem lies in the amz_cano_path function, which incorrectly replaces double slashes with a single slash.

Expected Behavior:
The library should correctly handle paths starting with a double slash and generate valid signatures for AWS requests without altering the path.

Proposed Solution:
To address this issue, I suggest modifying the amz_cano_path function to correctly handle paths starting with a double slash. The function should preserve the original path without replacing double slashes with a single slash.
This could be fixed by removing the following line fixed_path = re.sub('/+', '/', fixed_path)

Missing github tag for pypi version 1.0

The pypi version 1.0 has no git tag pushed to github, it would be awesome to keep that in sync. thanks!

Passing in keyword arguments leads to a TypeError

This if statement chain fails when keyword arguments are used for the region and service.

Code looks like this:

AWS4Auth(credentials.access_key, 
                   credentials.secret_key, 
                   region='us-east-1', 
                   service='aoss', 
                   session_token=credentials.token)

1.2.0 in PyPi differs from Github release

I installed 1.2.0 from PyPi, but it does not include #63.

Problems with requests 2.9.0

There seems to be a problem with requests 2.9.0, with 2.8.1 everything works fine, but with 2.9.0 I get this error:

TransportError(403, '{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'PUT\n/xxxx\n\ncontent-type:text/plain; charset=UTF-8\nhost:xxxxx\nx-amz-content-sha256:xxxxx\nx-amz-date:20151221T154751Z\n\ncontent-type;host;x-amz-content-sha256;x-amz-date\nxxxxx'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20151221T154751Z\n20151221/eu-west-1/es/aws4_request\nxxxxxx'\n"}')

Issues with Dynamic Credentials example

In the documentation we see the example of

>>> from requests_aws4auth import AWS4Auth
>>> from botocore.session import Session
>>> credentials = Session().get_credentials()
>>> auth = AWS4Auth(region='eu-west-1', service='es',
                    refreshable_credentials=credentials)
...

under the "Dynamic STS Credentials using botocore RefreshableCredentials" section.
This seems to be a defacto need for long running applications.
However, on running this fragment of code, we see the error:

raise TypeError(msg)
TypeError: AWS4Auth() takes 2, 4 or 5 arguments, 0 given

The implementation of dynamic credentials on my end is a direct copy and paste of the example given on the wiki so I don't understand where the issue is stemming from.

Include Content-MD5 in signed headers

Per the AWS docs, "The headers used for request signing are: content-md5, content-type, date, and anything that starts with x-amz-." It looks like content-md5 is excluded currently from the set of headers that is signed by requests-aws4auth, but it should be included. Our current workaround is just to pass the full set in include_hdrs kwarg, but would be good to have the defaults updated.

Regression in 1.1.0

Authentication has begun failing for us with multiple query strings because the 1.1.0 release (which is not tagged in GitHub) has introduced this commit from 5 years ago which appears to be a regression. The 1.0.1 branch did not include it

Any ideas how this happened / how it can be prevented moving forward?

Simple repro test:


qs_items = {'foo': ['bar', 'baz'], 'qux': ['3','2','1']}
qs_strings = []
for name in sorted(qs_items):
    vals = qs_items[name]
    for val in vals:
        qs_strings.append('='.join([name, val]))
    qs1 = '&'.join(qs_strings)

qs_strings = []
for name, vals in qs_items.items():
    for val in vals:
        qs_strings.append('='.join([name, val]))
    qs2 = '&'.join(sorted(qs_strings))

print(qs1)
print(qs2)

assert qs1 == qs2

foo=bar&foo=baz&qux=3&qux=2&qux=1
foo=bar&foo=baz&qux=1&qux=2&qux=3
Traceback (most recent call last):
  File "main.py", line 19, in <module>
    assert qs1 == qs2
AssertionError

EDIT: Ah I see though the commit is 5 years old, the merge just landed more recently.

UnicodeEncodeError when passing a non-ascii string in "data"

Sending a non-ascii request body using Python 2.7 fails when using requests-aws4auth. I thought it was a general requests bug at first (https://github.com/kennethreitz/requests/issues/3875) but it only happens with requests-aws4auth. I'm seeing this on Python 2.7.5 on centos 7.2 and macOS.

After some debugging, it seems to be triggered by string literals being forced to "unicode" in /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py.

from __future__ import unicode_literals

FIX/WORKAROUND: comment out that line.

The problem is requests doesn't seem to expect the HTTP request headers to contain unicode strings. Python 2.7 "unicode+str" weirdness causes request_headers + request_body to fail because request_body is already a binary(?) string.

Btw I don't think aws4auth should be doing an .encode('utf-8') -- it should already be "bytes", right? At least HTTPBasicAuth and S3Auth expect the client calling requests.put() to pass data already encoded to utf-8 bytes.

Finally, maybe this is still a bug in requests or python httplib.py? Should it allow unicode string headers, containing only ascii (or iso-8859-1?), and /usr/lib64/python2.7/httplib.py _send_output() should force msg to str before appending the request body?

Reproduction:

>>> import requests
>>> requests.__version__
'2.13.0'
>>> import requests_aws4auth
>>> requests_aws4auth.__version__
'0.9'
>>> AUTH=requests_aws4auth.AWS4Auth('testkey', 'secret', 'eu-west-1', 's3')
>>> requests.put('http://example.com/',headers={'Content-type':'text/plain; charset="UTF-8"'}, data=u'\u24B6\u24B7\u24B8\u24B9'.encode('utf-8'),auth=AUTH)

That should work, and it does when using requests.auth.HTTPBasicAuth or S3 V2 signature package awsauth.S3Auth. But requests-aws4auth gets exception:

>>> requests.put('http://example.com/',headers={'Content-type':'text/plain; charset="UTF-8"'}, data=u'\u24B6\u24B7\u24B8\u24B9'.encode('utf-8'),auth=AUTH)
!!!1 u'PUT / HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.13.0\r\nContent-type: text/plain; charset="UTF-8"\r\nContent-Length: 12\r\nx-amz-date: 20170215T040027Z\r\nx-amz-content-sha256: 7ec37a06579472c0743b58bd45af589cca817f65bbd8c6e528bc5e3092166396\r\nAuthorization: AWS4-HMAC-SHA256 Credential=john/20170215/eu-west-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=833120dd7cbe023d12c8bd24c6a746ba8ebcf8279346c0e58485e56c1a9ab5a5\r\n\r\n'
!!!2 '\xe2\x92\xb6\xe2\x92\xb7\xe2\x92\xb8\xe2\x92\xb9'
!!!3 u'PUT / HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.13.0\r\nContent-type: text/plain; charset="UTF-8"\r\nContent-Length: 12\r\nx-amz-date: 20170215T040027Z\r\nx-amz-content-sha256: 7ec37a06579472c0743b58bd45af589cca817f65bbd8c6e528bc5e3092166396\r\nAuthorization: AWS4-HMAC-SHA256 Credential=john/20170215/eu-west-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=833120dd7cbe023d12c8bd24c6a746ba8ebcf8279346c0e58485e56c1a9ab5a5\r\n\r\n\u24b6\u24b7\u24b8\u24b9'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 356, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python2.7/httplib.py", line 1020, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1054, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1016, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 865, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

The "!!!" lines are debugging output I added to /usr/lib64/python2.7/httplib.py _send_output()

        if isinstance(message_body, str):
            print('!!!1 '+repr(msg))
            print('!!!2 '+repr(message_body))
            print('!!!3 '+repr(msg + message_body.decode('utf-8')))
            msg += message_body

How to use with requests.post?

I may be brain dead, or just new at this, but how do you send a signed POST request?

Trying this:

class CredentialsProvider(object):

def __init__(self):
    self.creds = None
    self.sts = sts = boto3.client('sts')
    self.tz = get_localzone()
    self.session_num = 1

def auth(self):
    if not self.creds or datetime.now(self.tz) > self.creds['Expiration']:
        session_name = "sesh%d" % self.session_num
        self.session_num += 1
        self.creds = self.sts.assume_role(RoleArn='arn:aws:iam::XXXX:role/YYYYY', 
                                     RoleSessionName=session_name)['Credentials']
        self.auth = AWS4Auth(self.creds['AccessKeyId'],
                             self.creds['SecretAccessKey'],
                             'us-west-2', 'es',
                             session_token=self.creds['SessionToken'])
    return self.auth

...

    try:
        r = requests.post(url, data=json.JSONEncoder().encode(jsondata), auth=self.credentials_provider.auth())
    except Exception as e:
        print "HTTP exception for POST %s" % e

...

HTTP exception for POST call() takes exactly 2 arguments (1 given)

Works for get requests from the same host (python interactive) with
requests.get(endpoint, auth=awsauth)

I can't find any documentation on this either in the requests library or here.

use role credentials?

is there a way to use a role credential with requests-aws4auth?

instead of the following:

awsauth = AWS4Auth(args.awsaccesskey, args.awssecretkey, args.awsregion, 'es')

would there be a way to not send credentials into AWS4Auth or allow it to pick up instance metadata or (in my case specifically) AWS Lambda's role key info?

Port numbers are required if the port number is not 80/443

This doesn't work correctly:

headers['host'] = urlparse(req.url).netloc.split(':')[0]

It should check to see if the port number is either 80 or 443 and include it if not.

The request signature we calculated does not match the signature you provided.

This might seem strange but true. I am able to successfully query the AWS ES API for a simple query like this:

{
            "query": {
                "filtered": {
                    "filter": {
                        "range": {
                            "some_item.created_at": {
                                "gte": 1472473930000,
                                "lte": 1472548129171
                            }
                        }
                    }
                }
            },
            "size": 1000000000
        }

However, for complex queries, I get an error asking me to verify my AWS credentials.

New library is broken

The updates for the netloc is broken. It doesn't consider other ports besides 443/80 which makes it broken and computes the wrong signature.

Authentication fails for long URLs

In my testing an URL of 412 characters works, one of 458 characters fails.

No fancy characters, just ASCII encoded as UTF-8.

Tested on Windows 10 and Amazon Linux, Python 3.8. Version 1.0.1 of requests-aws4auth and 2.25.1 of requests.

Header x-amz-date vs. signing key amz-date

I started using this library yesterday, but was getting this error from AWS earlier today:

{"message":"Date in Credential scope does not match YYYYMMDD from ISO-8601 version of date from HTTP: \'20151124\' != \'20151125\', from \'20151125T154140Z\'."}

I believe what's happening is that AWS4Auth has amz_date set on AWS4SigningKey at initialization time, but the x-amz-date header is set at every request. If the AWS4Auth object is persisted across a change in dates (e.g., if it is a global object), the dates in the signing key and the header will eventually not match, resulting in the error above.

This may be by design (or maybe I'm misunderstanding the issue entirely), and I've resolved the issue by re-generating the AWS4Auth object for every connection, but I wanted to identify it in case this isn't by design.

Don't normalize S3 V4 paths (don't strip trailing / lone ".")

It's not a common or good S3 key name, but keys like "." and "//.." are valid keys and requests_aws4auth should generate the same signature as AWS S3.

# python
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> import requests_aws4auth
>>> AUTH=requests_aws4auth.AWS4Auth('AKIAXXXXXXXXXXFKXWMLQ', '4f6BgjVRgrfKIyk3fkUx4z29EZ9vCzOuVpC/hix8', 'us-east-1', 's3')
>>> r = requests.put('http://mybucket.example.com.s3.amazonaws.com/x/.', data='dot',auth=AUTH)
>>> r.text
u'<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAXXXXXXXXXXKXWMLQ</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256\n20170218T062045Z\n20170218/us-east-1/s3/aws4_request\n5700201b57931166eb49924611fbeb117099e7ac79dc7f9a14863bc6df684784</StringToSign><SignatureProvided>6ee791e3f664f9fab411ce7a11ed34b21cfbb827a415e2b1d4a66ac765e42a0d</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 37 30 32 31 38 54 30 36 32 30 34 35 5a 0a 32 30 31 37 30 32 31 38 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 35 37 30 30 32 30 31 62 35 37 39 33 31 31 36 36 65 62 34 39 39 32 34 36 31 31 66 62 65 62 31 31 37 30 39 39 65 37 61 63 37 39 64 63 37 66 39 61 31 34 38 36 33 62 63 36 64 66 36 38 34 37 38 34</StringToSignBytes><CanonicalRequest>PUT\n/x/.\n\nhost:mahbucke.s3.amazonaws.com\nx-amz-content-sha256:e392dad8b08599f74d4819cd291feef81ab4389e0a6fae2b1286f99411b0c7ca\nx-amz-date:20170218T062045Z\n\nhost;x-amz-content-sha256;x-amz-date\ne392dad8b08599f74d4819cd291feef81ab4389e0a6fae2b1286f99411b0c7ca</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 78 2f 2e 0a 0a 68 6f 73 74 3a 6d 61 68 62 75 63 6b 65 2e 73 33 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 39 32 64 61 64 38 62 30 38 35 39 39 66 37 34 64 34 38 31 39 63 64 32 39 31 66 65 65 66 38 31 61 62 34 33 38 39 65 30 61 36 66 61 65 32 62 31 32 38 36 66 39 39 34 31 31 62 30 63 37 63 61 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 37 30 32 31 38 54 30 36 32 30 34 35 5a 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 0a 65 33 39 32 64 61 64 38 62 30 38 35 39 39 66 37 34 64 34 38 31 39 63 64 32 39 31 66 65 65 66 38 31 61 62 34 33 38 39 65 30 61 36 66 61 65 32 62 31 32 38 36 66 39 39 34 31 31 62 30 63 37 63 61</CanonicalRequestBytes><RequestId>82D0203691B42F72</RequestId><HostId>+hs490EpTExGxdiLIodEzBDi+M7pcN9Ib+xGrI6bhrr4Qb0a6WzrVsqtJewvS+f80RxCTt0xl5Y=</HostId></Error>'

This fix seems to be simple, just remove posixpath.normpath() (which removes trailing / embedded dot and dot-dot), and stop removing duplicate "/".

But I guess that was added for a reason?

diff -u /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py~ /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py
--- /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py~	2017-02-15 19:17:02.000000000 +0000
+++ /usr/lib/python2.7/site-packages/requests_aws4auth/aws4auth.py	2017-02-18 00:18:15.425646504 +0000
@@ -288,7 +288,7 @@
                                    self.signing_key.secret_key is None):
             raise NoSecretKeyError
 
-        secret_key = secret_key or self.signing_key.secret_key
+        secret_key = secret_key or (self.signing_key and self.signing_key.secret_key) or ''
         region = region or self.region
         service = service or self.service
         date = date or self.date
@@ -604,7 +604,7 @@
         if '?' in fixed_path:
             fixed_path, qs = fixed_path.split('?', 1)
-        fixed_path = posixpath.normpath(fixed_path)
+#        fixed_path = posixpath.normpath(fixed_path)
-        fixed_path = re.sub('/+', '/', fixed_path)
+#        fixed_path = re.sub('/+', '/', fixed_path)
         if path.endswith('/') and not fixed_path.endswith('/'):
             fixed_path += '/'
         full_path = fixed_path

Update project to BETA or PRODUCTION/STABLE

Acording to List classifiers of PyPi I believe that we can move this project from ALPHA to BETA safely and plan to put the version to PROD if all the tests/validations are satisfied
https://pypi.python.org/pypi?%3Aaction=list_classifiers
Development Status :: 4 - Beta
Development Status :: 5 - Production/Stable

pypi package (0.5) will not install

The tarball is missing HISTORY.rst, so the package install fails.

setup automated testing

with travis or circleci or similar.

pip install from .tar.gz broken - HISTORY.md not packaged

setup.py reads in HISTORY.md, but that file is not present in the `1.2.2 source tarball distribution on PyPI. As a result, installing from source/tarball fails.

This can be verified by trying to install the .tar.gz currently on PyPI, or just looking at it and confirming that HISTORY.md is not included. It looks to me like this needs a MANIFEST.in file.

jantman@phoenix:pts/8:~/tmp$ python --version
Python 3.10.9
jantman@phoenix:pts/8:~/tmp$ pip --version
pip 22.3.1 from /usr/lib/python3.10/site-packages/pip (python 3.10)
jantman@phoenix:pts/8:~/tmp$ pip install requests-aws4auth -t lambda-deps/ --no-binary :all:
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting requests-aws4auth
  Using cached requests-aws4auth-1.2.2.tar.gz (25 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-nju4w6mc/requests-aws4auth_1127f02618df432787af867797df289a/setup.py", line 27, in <module>
          with codecs.open('HISTORY.md', 'r', 'utf-8') as f:
        File "/usr/lib/python3.10/codecs.py", line 906, in open
          file = builtins.open(filename, mode, buffering)
      FileNotFoundError: [Errno 2] No such file or directory: 'HISTORY.md'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Is there a urllib3 version of this library?

I want to use AWS ElasticSearch together with django-haystack but I'm bumping into problems trying to authenticate.

Haystack uses urllib3 for its web requests, and I can't find a way of integrating aws4auth easily (see also http://stackoverflow.com/questions/41147724/newconnectionerror-using-django-haystack-with-amazon-elasticsearch-as-backend ).

Is authenticating with aws4auth (or something similar) doable within this context?

Thanks!

No module named 'requests_toolbelt'

Since yesterday night all my Lambda functions on AWS that use requests-aws4auth can't run, with the error:

No module named 'requests_toolbelt'

Guess there is some dependency issue with the pip package?

Adding that library to my requirements files fixes the problem.

NOTE: the functions are deployed using the SAM CLI

You can add IVONA to 'Supported Services'

Hi,

Awesome project, thanks a bunch - you saved me a lot of hassle : )
I just wanted to let you know that you can add IVONA Speech Cloud API to list of supported services in README - I use it here and it works like a charm.

Modifying the content-type header breaks some integrations

The line [1] adds the charset to the Content-Type header and this breaks the integration with some services. In my tests, calling a Bedrock model was returning "The provided Content Type is invalid or nor supported for this model". After removing the line, the issue was gone.

[1] https://github.com/tedder/requests-aws4auth/blob/9e9e7bf25ad1962cd8dd77064216ee4cab8ca520/requests_aws4auth/aws4auth.py#L563C57-L563C72

Support for temporary STS credentials?

More of a question than an issue, really. Does this currently support STS temporary credentials? If not, any plans for that?

Mypy type annotations support

Hi, I'd like to request support for typing annotations that will work with mypy.

Thanks

Unicode headers name + SSL + big bodies

When a body is very big, using SSL, the unicode header names filled by aws4auth makes the requests crash:

OpenSSL.SSL.Error: [('SSL routines', 'SSL3_WRITE_PENDING', 'bad write retry')]

More info: https://www.bountysource.com/issues/27417596-ssl3_write_pending-error-from-urllib3-contrib-pyopenssl-sendall

I've solved making a wrapper that converts to str the header names:

    class AWS4AuthNotUnicode(AWS4Auth):
        def __call__(self, req):
            req = super(AWS4AuthNotUnicode, self).__call__(req)
            req.headers = {str(name): value for name, value in req.headers.items()}
            return req

Quoted reserved characters are unquoted when creating canonical query string

The AWS4Auth.amz_cano_querystring function unqoutes and then re-quotes the entire query string before parsing it. If the query string includes quoted reserved characters, then they get introduced into the query string as reserved characters changing its meaning.

from requests_aws4auth import AWS4Auth
from urllib.parse import quote

# Ampersand (&)
q = quote('a&b')
AWS4Auth.amz_cano_querystring(f'foo={q}&bar=1')

# result: b=&bar=1&foo=a
# expected: bar=1&foo=a%26b


# Plus (+)
q = quote('a+b')
AWS4Auth.amz_cano_querystring(f'foo={q}&bar=1')

# result: bar=1&foo=a%20b
# expected: bar=1&foo=a%2Bb


# Space
q = quote('a b')
AWS4Auth.amz_cano_querystring(f'foo={q}&bar=1')

# result: foo=a
# expected: bar=1&foo=a%20b

Type Error when using aws4auth for authenticating opensearch service in langchain.

Code

import boto3
from langchain.vectorstores import OpenSearchVectorSearch

service = "es" 
region = "ap-southeast-1"
credentials = boto3.Session().get_credentials()

awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, 
                  region, service, session_token=credentials.token)

docsearch = OpenSearchVectorSearch(
    index_name=index_name,
    embedding_function=embeddings_endpoint,
    opensearch_url= my_url,
    http_auth=awsauth,
    use_ssl=False,
    verify_certs=False,
    ssl_assert_hostname=False,
    ssl_show_warn=False    
)

Error

TypeError                                 Traceback (most recent call last)
File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py:262, in Urllib3HttpConnection.perform_request(self, method, url, params, body, timeout, ignore, headers)
    261     if isinstance(self.http_auth, Callable):  # type: ignore
--> 262         request_headers.update(self.http_auth(method, full_url, body))
    264 response = self.pool.urlopen(
    265     method, url, body, retries=Retry(False), headers=request_headers, **kw
    266 )

TypeError: AWS4Auth.__call__() takes 2 positional arguments but 4 were given

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
Cell In[29], line 1
----> 1 docsearch.add_documents(documents=text)

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/langchain_core/vectorstores.py:119, in VectorStore.add_documents(self, documents, **kwargs)
    117 texts = [doc.page_content for doc in documents]
    118 metadatas = [doc.metadata for doc in documents]
--> 119 return self.add_texts(texts, metadatas, **kwargs)

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/langchain_community/vectorstores/opensearch_vector_search.py:411, in OpenSearchVectorSearch.add_texts(self, texts, metadatas, ids, bulk_size, **kwargs)
    392 """Run more texts through the embeddings and add to the vectorstore.
    393 
    394 Args:
   (...)
    408     to "text".
    409 """
    410 embeddings = self.embedding_function.embed_documents(list(texts))
--> 411 return self.__add(
    412     texts,
    413     embeddings,
    414     metadatas=metadatas,
    415     ids=ids,
    416     bulk_size=bulk_size,
    417     **kwargs,
    418 )

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/langchain_community/vectorstores/opensearch_vector_search.py:370, in OpenSearchVectorSearch.__add(self, texts, embeddings, metadatas, ids, bulk_size, **kwargs)
    364 _validate_aoss_with_engines(self.is_aoss, engine)
    366 mapping = _default_text_mapping(
    367     dim, engine, space_type, ef_search, ef_construction, m, vector_field
    368 )
--> 370 return _bulk_ingest_embeddings(
    371     self.client,
    372     index_name,
    373     embeddings,
    374     texts,
    375     metadatas=metadatas,
    376     ids=ids,
    377     vector_field=vector_field,
    378     text_field=text_field,
    379     mapping=mapping,
    380     max_chunk_bytes=max_chunk_bytes,
    381     is_aoss=self.is_aoss,
    382 )

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/langchain_community/vectorstores/opensearch_vector_search.py:118, in _bulk_ingest_embeddings(client, index_name, embeddings, texts, metadatas, ids, vector_field, text_field, mapping, max_chunk_bytes, is_aoss)
    115 mapping = mapping
    117 try:
--> 118     client.indices.get(index=index_name)
    119 except not_found_error:
    120     client.indices.create(index=index_name, body=mapping)

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/client/utils.py:181, in query_params.<locals>._wrapper.<locals>._wrapped(*args, **kwargs)
    178         if v is not None:
    179             params[p] = _escape(v)
--> 181 return func(*args, params=params, headers=headers, **kwargs)

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/client/indices.py:255, in IndicesClient.get(self, index, params, headers)
    252 if index in SKIP_IN_PATH:
    253     raise ValueError("Empty value passed for a required argument 'index'.")
--> 255 return self.transport.perform_request(
    256     "GET", _make_path(index), params=params, headers=headers
    257 )

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/transport.py:446, in Transport.perform_request(self, method, url, params, body, timeout, ignore, headers)
    444     # raise exception on last retry
    445     if attempt == self.max_retries:
--> 446         raise e
    447 else:
    448     raise e

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/transport.py:409, in Transport.perform_request(self, method, url, params, body, timeout, ignore, headers)
    406 connection = self.get_connection()
    408 try:
--> 409     status, headers_response, data = connection.perform_request(
    410         method,
    411         url,
    412         params,
    413         body,
    414         headers=headers,
    415         ignore=ignore,
    416         timeout=timeout,
    417     )
    419     # Lowercase all the header names for consistency in accessing them.
    420     headers_response = {
    421         header.lower(): value for header, value in headers_response.items()
    422     }

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/opensearchpy/connection/http_urllib3.py:279, in Urllib3HttpConnection.perform_request(self, method, url, params, body, timeout, ignore, headers)
    277     if isinstance(e, ReadTimeoutError):
    278         raise ConnectionTimeout("TIMEOUT", str(e), e)
--> 279     raise ConnectionError("N/A", str(e), e)
    281 # raise warnings if any from the 'Warnings' header.
    282 warning_headers = response.headers.get_all("warning", ())

ConnectionError: ConnectionError(AWS4Auth.__call__() takes 2 positional arguments but 4 were given) caused by: TypeError(AWS4Auth.__call__() takes 2 positional arguments but 4 were given)