mhart / aws4 Goto Github PK

Hi Michael,

Have you tested against Cognito User Pools by chance? I am trying to piece together a working call, but keep getting a 504 Gateway Timeout error. I'm guessing on x-amz-target and content-type headers because there's no docs yet on specific values for these parameters.

{ host: 'cognito-idp.us-east-1.amazonaws.com', path: '/', service: 'cognito-identity', body: '{"PoolName":"First Test Pool"}', headers: { 'Content-Type': 'application/x-amz-json-1.1', 'X-Amz-Target': 'CognitoIdentityServiceProvider.CreateUserPool', Host: 'cognito-idp.us-east-1.amazonaws.com', 'Content-Length': 30, 'X-Amz-Date': '20160819T223312Z', Authorization: '...', Signature='...' }, method: 'POST' }

Getting following response from AWS API Gateway. Things were fine before but now just the host for the service has changed. Just with that change in request, following error has start poping up.

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n\'GET\n/stage/v1/adproduct/a03e8a66-55e9-4530-968e-921df2ed826c/result\n\ncontent-type:application/json\nhost:example.com\nx-amz-date:20160919T154421Z\n\ncontent-type;host;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b856\

Any inputs would be super helpful.

CC @chiranjitdutta1985 @ukils

I'm using this project to authorize requests to an API Gateway service we have. There is an issue where the X-Amz-Date is expiring after five minutes, which is expected. But what is not expected is for the signature to not get renewed. Subsequent request past the five minute mark are continued to be signed with the expired date, resulting in Signature Expired errors from the service.

Hi,

I use sdk as below:
var opts = {service: 's3', region: 'us-east-1', path: '/xxx/xxx/xxx', method: 'PUT'}
var sign = aws4.sign(opts1, {accessKeyId: AWSAccessKeyId, secretAccessKey: YourSecretAccessKeyID})

But I use postman and set details from JSON.stringify(sign) as below picture, then get error
"The request signature we calculated does not match the signature you provided. Check your key and signing method."

but method:'GET' is work

Does anyone know this problem?
Please let me know if I mistake the usage.
Thanks very much

Sorting of the query string variables breaks signing as its not part of the spec:
http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

The keys should be sorted but not the values, so this like breaks thinks:
https://github.com/mhart/aws4/blob/master/aws4.js#L226

Simply removing the sort will fix the issue.

Hey,

I noticed that the canonicalHeaders function uses Object.keys. Object.keys ignores inherited properties. The Fetch standard for one, or at least every implementation, takes inherited headers into account when creating the request. That discrepancy will mean some headers will not properly be signed by Aws4. You might want to switch to using the classic for (var key in headers) style.

Cheers

I'm using webpack to bundle aws4, and when I set a path other than /, I get the following error:

Uncaught TypeError: querystring.escape is not a function

The stacktrace is pointing to

I tried adding the following node config to webpack (to see if it would polyfill), but it didn't seem to change anything:

node: {
  querystring: true,
}

Hello, the request is freezing, simple do nothing:

var opts = {
host: config.elasticSearchHostName,
method: 'GET',
region: 'eu-west-1',
service: 'es',
path: '/calendars/_search',
headers: {
'Host': config.elasticSearchHostName,
'Accept': 'application/json',
'X-Amz-Date': moment().format()
},
body: JSON.stringify({
query: {
match_phrase: {
calendar_flipped_class_id: "/blended/api/v1/class/3723/"
}
},
sort: { calendar_date: { order: "asc" } }
})
}

aws4.sign(opts, {
accessKeyId: config.accessKeyId,
secretAccessKey: config.secretAccessKey
}
);

httpClient.request( opts, function(res) { res.pipe(process.stdout) }).end()

Can you please help me?

I am getting this error when trying to use this in angular cli. I searched for the error and found that apparently node crypto is switched off in angular https://github.com/angular/angular-cli/issues/1548.

I tried the workaround at the bottom of this issue (which is a bit of a hack) and got past the crypto error but now i get HomeComponent.html:1 ERROR TypeError: querystring.escape is not a function.

I feel like I am missing something as other people seem to be using this with angular so I just wanted to check with you if I am being stupid. I am going to go back to using AWS.Signers.V4 for now but your package seems much cleaner.

Stack from the cypto error:
HomeComponent.html:1 ERROR TypeError: crypto.createHmac is not a function
at hmac (aws4.js:11)
at RequestSigner.signature (aws4.js:182)
at RequestSigner.authHeader (aws4.js:173)
at RequestSigner.sign (aws4.js:144)
at Object.aws4.sign (aws4.js:331)
at ApiGatewayService.apiCall (api-gateway.service.ts:59)

Stack from querystring error:
HomeComponent.html:1 ERROR TypeError: querystring.escape is not a function
at eval (aws4.js:237)
at Array.reduce ()
at RequestSigner.canonicalString (aws4.js:232)
at RequestSigner.stringToSign (aws4.js:196)
at RequestSigner.signature (aws4.js:188)
at RequestSigner.authHeader (aws4.js:173)
at RequestSigner.sign (aws4.js:144)
at Object.aws4.sign (aws4.js:331)
at ApiGatewayService.apiCall (api-gateway.service.ts:59)

See the list of hostnames on http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region.
Some regions are only accessible on a hostname of this form:

s3-eu-west-1.amazonaws.com

This kind of hostnames breaks the 'matchHost' method, as it looks for '.' characters to determine parts of the hostname.
In this case, both the region and the service are tied together as 's3-eu-west-1'.
This breaks functionality as this.service is now no longer 's3' and this.region is 's3-eu-west-1', which is not valid.

As a workaround, both the service and the region can be specified by the calling code, which makes the issue less important. Still, inferring service and region from the URL is not working properly for AWS S3.

Hi there,

I have the following code working just fine (upload to S3):

const fs = require("fs");
const aws4 = require("aws4");
const https = require("https");

const hash = aws4.sign({
    service: "s3",
    region: "eu-central-1",
    method: "PUT",
    path: "/my-bucket/image.jpg",
    headers: {
        "Content-Type": "image/jpeg"
    },
    body: fs.readFileSync("./image.jpg")
}, {
    accessKeyId: "my-access-key",
    secretAccessKey: "my-secret-key"
});

var req = https.request({
    hostname: "s3-eu-central-1.amazonaws.com",
    port: 443,
    method: "PUT",
    path: "/my-bucket/image.jpg",
    headers: hash.headers
});

fs.createReadStream("./image.jpg").pipe(req);

As I understand correctly, the whole file that is being uploaded needs to be added to the signature. What worries me is that line:

body: fs.readFileSync("./image.jpg")

What if image.jpg is a 500MB file which is loaded into memory just to calculate the signature? Is there any other possibility to upload a file to S3 in a more performant way?

Hi,

Am facing an InternalFailure error with the aws4 node.js

The code that am using pretty simple.

var aws4 = require('aws4');
http = require('request')
var CREDS = {accessKeyId: 'Access_Key',
secretAccessKey: 'Secret_Key'}

var options = {
service : 'support',
method: 'POST',
headers: {
'x-amz-target': 'AWSSupport_20130415.DescribeCases',
'content-type': 'application/x-amz-json-1.1'
},
body: '{}'
};

var sigs = aws4.sign(options, CREDS);
console.log(sigs);
sigs['url'] = 'https://support.us-east-1.amazonaws.com/';

http(options, function (error, response, body) {
console.log('-------- body --------' + body );
if (error) throw new Error(error);
});

The error that i get: {"__type":"InternalFailure"}

Can you tell me whats wrong with this code, when I try to use the python version of this code it works fine without any issues.

I can sign a GET request successfully, but I am unable to perform a PATCH. The OPTIONS pre-flight request appears to have all the extra headers it needs to inform the AWS server about. The OPTIONS response appears to show all the headers necessary to make the PATCH request, but the PATCH response gives me InvalidSignatureException:
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://gdavis.dev' is therefore not allowed access. The response had HTTP status code 403.

I am using the gateway generated javascript SDK. I attached some images showing the requests and responses for OPTIONS and PATCH. Let me know if there is any other information that may be helpful. I tried removing the body to see if I could reach the gateway, but I keep getting error 415 when I try that. For some reason I can't figure out how to make content-type: application/json when there is no body even though application/json is the set as the default. I can forcefully add it in the additional params, but then the request shows multiple content-type keys and I still get 415.

First off, great job with your awesome library!

I'm having an issue with POST requests to cognito-sync.

const body = JSON.stringify({key1: 'foo', key2: 'bar'});
const params = {
    service: 'cognito-sync',
    region: 'eu-west-1',
    host: 'cognito-sync.eu-west-1.amazonaws.com',
    path: "/",
    body,
    method: 'POST',
    headers: {
      'X-Amz-Target': 'com.amazonaws.cognito.sync.model.AWSCognitoSyncService.RegisterDevice',
      'Content-Type': 'application/x-amz-json-1.0',
      "Content-Length": body.length
    },
}

This returns the classic error: The request signature we calculated does not match the signature you provided.

aws4 seems to be generating headers w/o much fuss

{
    Accept: 'application/json, text/plain, */*',
    'Content-Type': 'application/x-amz-json-1.0',
    'X-Amz-Target': 'com.amazonaws.cognito.sync.model.AWSCognitoSyncService.RegisterDevice',
     'X-Amz-Security-Token': <A ATOKEN>,
    'X-Amz-Date': '20171030T105619Z',
    Authorization: 'AWS4-HMAC-SHA256 Credential=ASIAIU7OT4UWIAAPI5OA/20171030/eu-west-1/cognito-sync/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-target, Signature=<A SIGNATURE>'
}

Then, I call axios with the data as JSON.parse(body)

My guess is that I'm messing something up with passing parameters to the library, but I'm not sure exactly what is wrong... The source code is pretty copious and well documented, so I'm reluctant to go into making a pull request yet for something that is not broken if I'm the one messing up. Otherwise, the library works great for the API gateway. Many thanks :-)

I'm bundling the latest version of aws4 with the latest version of webpack for use in the browser. I'm receiving the following error:

f.escape is not a function

Elasticsearch domains are of the format <domain>.<region>.es.amazonaws.com, so it seems that using service: "es" does not work since this yields es.<region>.amazonaws.com and I get the error:

Host: es.us-east-1.amazonaws.com. is not in the cert's altnames: DNS:*.us-east-1.es.amazonaws.com

So I change it to use the actual host of my domain instead:

axios(aws4.sign({
    host: process.env.ES_ENDPOINT,
    method: "post",
    url: `https://${process.env.ES_ENDPOINT}/foobot/foobot`,
    data,
}))

Now I get this error from AWS http response:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/foobot/foobot

host: <ES-HOST>.us-east-1.es.amazonaws.com
x-amz-date:20160726T211903Z
x-amz-security-token:<TOKEN>

host;x-amz-date;x-amz-security-token
<TOKEN>'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20160726T211903Z
20160726/us-east-1/es/aws4_request
<TOKEN>'

Hi! Thanks for the commits regarding the path escaping.

The signing still fails if the path contains a :. It gets transformed into %3A earlier and then on line 219:

path.push(encodeRfc3986(querystring.escape(piece)))

querystring.escape turns it into %253A.

So signing calls to e.g. http://domain.com/product/foo:123 fails.

I have a problem uploading an image using axios. This code runs perfect on IOS but on Android throws the error Multipart != Application/JSON.
If someone had the same error and knows how to fix it is welcome.
Thanks

var photo = {
uri: data.image_url,
type: "image/jpeg",
name: "photo.jpg"
};
formData.append("photo", photo);
}

formData.append("route_id", data.gym_id);

  let url = `${config.protocol}://${config.domain}:${
    config.port
  }/webapp/createRoute`;
  
  axios
    .post(url, formData, {
      headers: {
        Accept: "application/json",
        "Content-Type": "multipart/form-data"
      }
    });

This looks awesome. What is the best way to implement this outside of Node.JS? Say I simply want to execute the entire signing process in a web page, then pass the signed URL to a proxy (in a SharePoint app), what's the easy way to do that? Any help would be appreciated.

I should add that I have a special case for calling a service that is not an AWS service, but supports AWS V4 signatures.

I am not sure if I am doing this wrong but I am able to get the signing of GET requests to work just fine. I am unable to get PUT and DELETE requests to work.

On the failing requests I get the following response header.

x-amzn-ErrorType:InvalidSignatureException

Here is the example of a working GET

GET /dev/whitelist HTTP/1.1
Host: crh3zvjyzb.execute-api.us-east-1.amazonaws.com
Connection: keep-alive
Authorization: AWS4-HMAC-SHA256 Credential=ASIAIQMWNYUKZ7WXQ4HA/20160422/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token, Signature=e88f6d15de04f33a9f9e2f0a620f36c1248ad78bcf0ebabb4e32d3dc41c08079
Origin: http://forecasts-admin.localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
Content-Type: application/json
Accept: application/json, text/plain, /
X-Amz-Security-Token: FQoDYXdzEDQaDKDNSamcVHL8LibnlSKnAtiO61HAbJt1B+jagfq7jezCvB6WD07MDhqobTIS8XicEduhCOV+GqDPSf2H+odacL+8IzVICVdcNwGM2Ue40DqR7PiW9UMNYBdVhIbBNRcRvpDxEOQixp5J89GiAf3xkzJnZcXuUT77qEoQ148XHd1z9nzW0N/VUrJOMvJLt8yY09/rxMj9u+ZFVUDnvQ3JGvuGYvxi8GIVhZ/UcXbKpddLSDWYsZjTOlb1NwLwvfTMS/5TpoAeYW7E313kI/HNrO/GFiiFZRaJTls5Aj2K1xuOpWOKKJjhtX2GfaEQkEy8h545gpfseooLxJKVzxo1KttYQFRU5fkoeBGh0+WVx/Nx3ZPUbYZxGT2+caNvkCY2GxDSNBEkt9ZJkDJjng9SujErACrunVIo8+PpuAU=
X-Amz-Date: 20160422T184900Z
Referer: http://forecasts-admin.localhost:8080/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,es;q=0.6,fr-CA;q=0.4,fr;q=0.2,es-419;q=0.2,en-CA;q=0.2

And of a failing DELETE

DELETE /dev/whitelist/c21ad120-0670-11e6-a64b-4937ef9e0275 HTTP/1.1
Host: crh3zvjyzb.execute-api.us-east-1.amazonaws.com
Connection: keep-alive
Content-Length: 2
Authorization: AWS4-HMAC-SHA256 Credential=ASIAIQMWNYUKZ7WXQ4HA/20160422/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token, Signature=b9f923d12daf049e4e744b7dea7745e66128489b9c16ab98351be98634ee6014
Origin: http://forecasts-admin.localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
Content-Type: application/json
Accept: application/json, text/plain, /
X-Amz-Security-Token: FQoDYXdzEDQaDKDNSamcVHL8LibnlSKnAtiO61HAbJt1B+jagfq7jezCvB6WD07MDhqobTIS8XicEduhCOV+GqDPSf2H+odacL+8IzVICVdcNwGM2Ue40DqR7PiW9UMNYBdVhIbBNRcRvpDxEOQixp5J89GiAf3xkzJnZcXuUT77qEoQ148XHd1z9nzW0N/VUrJOMvJLt8yY09/rxMj9u+ZFVUDnvQ3JGvuGYvxi8GIVhZ/UcXbKpddLSDWYsZjTOlb1NwLwvfTMS/5TpoAeYW7E313kI/HNrO/GFiiFZRaJTls5Aj2K1xuOpWOKKJjhtX2GfaEQkEy8h545gpfseooLxJKVzxo1KttYQFRU5fkoeBGh0+WVx/Nx3ZPUbYZxGT2+caNvkCY2GxDSNBEkt9ZJkDJjng9SujErACrunVIo8+PpuAU=
X-Amz-Date: 20160422T184907Z
Referer: http://forecasts-admin.localhost:8080/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,es;q=0.6,fr-CA;q=0.4,fr;q=0.2,es-419;q=0.2,en-CA;q=0.2

I passed the query string 'qs' in the options then i signed my request for the API Gateway, but it still fails, turns out that aws4.sign(opts) does not take into account the opts.qs parameter for the querystring, so i had to compile the opts.path and opts.url if i want to add a query string before signing the request:

    // add the query string to the path name and the url
    query = {"p1": v1, "p2": v2};

    opts.path = opts.path + '?' + qs.stringify(query);

    opts.url =  urlParse.format({
      host: host,
      protocol: protocol,
      pathname:  opts.path,
      query: query
    });

    // Sign the new request
    AWS4.sign(opts, {
      accessKeyId: configuration.accessKeyId,
      secretAccessKey: configuration.secretAccessKey
    });

    return rb.get(opts)

Hi! First, thanks for creating this module, it has made my development life a lot easier. For such a valuable function, I'm surprised its not in the aws-sdk (or maybe it is, and I just can't find it :) )

I recently ran into an issue where I was signing HTTPS requests that were attempting to push data into AWS Elasticsearch from a lambda. My policy restricted POSTs and PUTs to the role my lambda was operating under. So far so good.

I used the library to sign requests, but kept getting HTTP 403 responses, with a message saying "The request signature we calculated does not match the signature you provided..."

After playing with what I had in my request options for a bit and combing through logs, I found that I had explicitly set the "port" property of my request options to 443 prior to signing it.

When I removed the port option, the request was properly signed and my index requests started returning HTTP 200.

From my logs, it looked like the aws4 lib was including the port as part of the "String to sign", but AWS was excluding it.

It might be the case that since 443 is the standard https port, AWS erroneously drops the port from the string to sign, or maybe Node.js drops it from the actual request, or maybe aws4 problematically includes it in the host before signing... I haven't delved quite that far yet. Wondering if you've run into this situation before?

Hi @mhart
I'd like to call AWS services via https but rather than using fixed AWS credentials I want to use the Cognito Identity credentials of the currently logged in Cognito User.
How can this be done?

I'm unsure if this is related to; #23 as I was unclear how it was actually resolved.

I can perform GET requests just fine which are authorised correctly, however, as soon as I try and do POST or PUT (I would assume DELETE too), I get a 403 InvalidSignatureException error. These hits also don't get logged to CloudWatch although the OPTIONS request seems to go through fine (and gets logged).

I'm using jQuery to make my requests like so;

var API = {
        /**
         * Make an XHR request, butchered from https://github.com/remy/libraries
         * @param  {string}   type     The request type (GET, POST, PUT)
         * @param  {string}   url      The URL to call
         * @param  {mixed}   opts     Callback function, or set of options
         * @param  {Function} callback Callback function, should take 2 params, err and repsonse
         * @return {XHRHttpRequest}            [description]
         */
        request: function (type, url, callback) {
            var host = CONFIG.API_ENDPOINT;
            var path = "/" + CONFIG.API_ENV + url;

            // Sign the request
            var opts = aws4.sign({
                host: host,
                path: path,
                service: 'execute-api',
                region: CONFIG.REGION
            },
            {
                accessKeyId:CONFIG.ACCESS_KEY,
                secretAccessKey: CONFIG.SECRET_KEY,
                sessionToken: CONFIG.SESSION_TOKEN
            });

            // Because we can't actually send the host header (chrome throws an error)
            delete opts.headers.Host;

            return $.ajax({
                type: type,
                url: "https://" + opts.host + opts.path,
                headers: opts.headers
            });
        }
    };

    API.get = API.request.bind(this, 'GET');
    API.post = API.request.bind(this, 'POST');
    API.put = API.request.bind(this, 'PUT');

Here's the Request and Response I'm sending / receiving;

General

Request URL:https://******/
Request Method:PUT
Status Code:403 Forbidden
Remote Address: *.*.*.*:443

Request Headers

Accept:*/*
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Authorization:AWS4-HMAC-SHA256 Credential=ASIAJ2FOGYMTXFJOUQQA/20160830/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token, Signature=bdb46e463dfa55650541852d2657eddb02d554db3720904358102efe28b70adf
Cache-Control:no-cache
Connection:keep-alive
Content-Length:0
Content-Type:application/json
Pragma:no-cache
Origin:http://bitc.dev
Referer:http://bitc.dev/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
X-Amz-Date:20160830T163359Z
X-Amz-Security-Token:FQoDYXdzEC8aDLz6H08kKi8EUa+PhCL1BMCs+sZpuhnQL8nSPtxV8prSuWX3Sq0Lks8/MQfbOWQun0Qv4O+/TWQ4lpVNkZi+33apxAAutU0xN3v4xVIWAmd94QlU+xjplHsgxYblt7eVhqCNZ2T9ZgXCthtVJsJNLU6/CM0eQHyHpOotUgqFDNcL1Y7rXYTXMZZJCBm8OnKDj2lgjXeE5VIbXwKNrAmqQvzmeI4upEyuxcfiyZ4Nxuwx+QoecMIn5iGJ+E/p7s8f81uXtD83Y41SswU869mn8syvUGfGRqKYth/dihRImYkq0Aoohke9xjfZ/C9JsXr2cnp0PENBuxLrGHNc3d8wUY5ZvktYavw78fZbKc4kHQb1wM1VMDr6FQzrnjjNzvUr1C35YIGw5hoHYm+U1AJc7bztqFB1gjNc6rghJnpcG3mTgUhkFje+rIeshHteKUgWX6TW6fpj814evibExvMQSTfkAa6kGlXl5GjqW1hwTJ9i27ixSCMCrRnb/jO6YbjqCVkrWkqSg8IqxWEEkS2C4Dcu2CYsuN+LP0/KZJ6DxSfur27LnppVcKE7B+NwwidxM8l4adU4/uI/W9XQYWtSLCdbj10OayI4GzOQxpeOhDPQdPlPI5OnBMIRgVEHnuDDLuT3pfgL18d2sqES3W+SWxbawR/Jm7Z1bClo+Kiry2epSXn550Lr3GXr3UzBSJvJOzYa5v77QZadoDCrGX2g0yqpMypbWeAu5zckKV/ePl14lbGW/65WT0rC9SxbPVrdhRoKUJQZSpwz03izg9IfpoY4xiyMqgviueqEtYn5TbWXT4u5U3Z2yHQdg/SIdxORaHpIgSlPLfP359zDZpBBJ4IkYEnMKKmRlr4F

Response Headers

Connection:keep-alive
Content-Length:1552
Content-Type:application/json
Date:Tue, 30 Aug 2016 16:43:49 GMT
Via:1.1 c77b51ad135b3319a54e2e40de778962.cloudfront.net (CloudFront)
X-Amz-Cf-Id:UqkPTl73hkNEghrrxVBI1czDjZY2TNd3fvBWCcDHRtS1lGpSDb15yA==
x-amzn-ErrorType:InvalidSignatureException
x-amzn-RequestId:ed00bbcd-6ed0-11e6-b916-6db2f5be0977
X-Cache:Error from cloudfront

I have tried with setting with and without a 'body' value on the request with the same error.

Hi and thanks a lot (!) for this library. It's super helpful. We've started working with MQTT and AWS' IoT services. In this context we found some limitations in the current version of aws4.

The IoT service knows at least two distinct "service" name signatures need to be generated for: iotdata and iotdevicegateway. However, the host name for both services is always "*.iot.us-east-1.amazonaws.com" (note the iot). The magic of aws4 to build a host name out of a given service name and vice versa fails for this case. I'm using the popular request module and that seemingly relies on this magic, failing to properly sign my requests. The protocols for IoT are described here: http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html -- although it misses any hint on the iotdata service.

I've tried to specify the host name and the service name manually, but without luck.

So finally I've ended up in using aws4 manually to sign my request options, and then forward that to request:

    const cred = credentials = new AWS.EnvironmentCredentials("AWS");
    const topic = "my/topic";
    const body = JSON.stringify({ foo: "bar" });
    const opts = {
        sign_version : 4,
        service: "iotdata",
        region: this.region,
        host: "<endpoint>.iot.us-east-1.amazonaws.com"
        path: `/topics/${encodeURIComponent(topic)}?qos=1`,
        body: body,
        headers: {
            "content-type": "application/javascript"
        },
    };
    // sign the request manually here
    aws4.sign(opts, {
        accessKeyId: cred.accessKeyId,
        secretAccessKey: cred.secretAccessKey,
        sessionToken: cred.sessionToken
    });

    // fire it up
    request.post(`https://${opts.host}${opts.path}`, opts, response => { ... }));

Though this is probably also a bug in the request module, this seems to be something that could be handled in aws4 by automatically mapping iotdata and iotdevicegateway to the actual iot hostname part and vice versa. iotdata is used for MQTT (<endpoint>.iot.us-east-1.amazonaws.com/topics) as well as for the device shadows (<endpoint>.iot.us-east-1.amazonaws.com/things), iotdevicegateway is used for the WebSocket requests (<endpoint>.iot.us-east-1.amazonaws.com/mqtt). Any thoughts?

Thanks for the great library! Unfortunately, this module can't be used with React Native, because core Node JS modules like crypto and querystring aren't accessible from React Native apps.

I've made a fork of this library here: https://github.com/aakashns/aws4-react-native , modified to work with React Native apps. The core Node JS module querystring has been replaced by querystring-browser, and crypto has been replaced by a standalone javascript file crypto.js generated using browserify. There are no other code changes.

I considered adding a pull request here, but decided against it because the polyfill for the crypto module is 500kb in size, which is probably too big a file to send down with an npm module even though it's not required for the common case (Node JS). So, I'm just leaving the links out here for anyone who comes looking.

React Native compatible version: https://www.npmjs.com/package/aws4-react-native (source code)

According to http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html, S3 also supports sig v4 signing.

I had a situation where I needed to POST to a custom API using AWS Signature v4, I used your library but it only was able to get it to work when I suppressed the signing of the 'content-length' header.

I did this by forking and deleting the two lines in aws4.js that had to do with applying the content-length header. I just wanted to let you know as I'm uncertain why this is happening (maybe intermediary filtering?) or if others were running into requirements to restrict the signed headers. I'm happy to spend some time adding a feature if you had any thoughts about how we could integrate this functionality back into your library.

Thanks for your thoughts, appreciate your work on this!

Swapping node's crypto for calvinmetcalf/native-crypto would let aws4 seamlessly use the WebCrypto API where it's available (which is basically everywhere now).

There's some pretty significant performance gains to be had but aws4 would have to be rewritten to handle the Promises returned by native-crypto.
Or if support for older browsers/node are not a big priotity, it would be real simple to rewrite using aync/await . Transpiling on publish gets rid of this issue but would require adding a build step, not that much work but still a change.

Signature for s3 does not work. Aws gives me "The request signature we calculated does not match the signature you provided. Check your key and signing method."

The cause seems to be that this module signs "s3.ap-northeast-1.amazonaws.com", whereas it should sign "s3-ap-northeast-1.amazonaws.com"

Hello, thx so much for this module, i'm having trouble for simple request.

here is my options

opts = { method: 'POST',
  host: 'agcod-v2-eu-gamma.amazon.com',
  path: '/sping/CreateGiftCard',
  'content-type': 'application/x-www-form-urlencoded; charset=utf-8',
  service: 'AGCODService',
  region: 'eu-west-1' }

after signing opts looks like

{ method: 'POST',
  host: 'agcod-v2-eu-gamma.amazon.com',
  path: '/sping/CreateGiftCard',
  'content-type': 'application/x-www-form-urlencoded; charset=utf-8',
  service: 'AGCODService',
  region: 'eu-west-1',
  headers:
   { Host: 'agcod-v2-eu-gamma.amazon.com',
     'X-Amz-Date': '20160627T112940Z',
     Authorization: 'AWS4-HMAC-SHA256 Credential=AKIAJBBXKNIY3ZLOSBTA/20160627/eu-west-1/AGCODService/aws4_request, SignedHeaders=host;x-amz-date, Signature=b6972db382a3f4137ae792241a56677a4c48fabd8a32d21d6bcf875f21b21724' } }

but this is what amazon respond making http request with https.request(opts, function(res) { res.pipe(process.stdout) }).end()

<InvalidSignatureException>
  <Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/sping/CreateGiftCard

host:agcod-v2-eu-gamma.amazon.com
x-amz-date:20160627T112940Z

host;x-amz-date
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20160627T112940Z
20160627/eu-west-1/AGCODService/aws4_request
1bf2d911cc922ef05917b27ce6b9eba03ff51f6cd60f08113a16f5ea7826ac0f'
</Message>
</InvalidSignatureException>

any idea on this ?

thanks

You should be able to use the AWS_PROFILE environment variable to specify a credentials profile to set the access/secret/session information for signing.

This regular expression will work for ElasticSearch… Would you be open to this as a PR?

/^(?:[^\.]+\.)?([^\.]+)\.?([^\.]*)\.amazonaws\.com$/

Example ElasticSearch hostname:-
search-next-search-aaaa00aaaa0aaa0aaaaaaa0aaa.us-east-1.es.amazonaws.com

On this line:-
https://github.com/mhart/aws4/blob/master/aws4.js#L46

Diff:-
master...matthew-andrews:patch-1

I'm signing some requests to use elastic search. I know the configuration api requests are signed with v4, but unless you have an IP based acl you need to identify yourself for access to elastic search itself.

http-aws-es works great with the elasticsearch nodejs library, but that library is 4+ MB for basically an http client. I figured it was worth figuring this out for my lambda functions.

This works! So you can add it to the list

import AWS from 'aws-sdk'
import aws4 from 'aws4'
import bole from 'bole'
import got from 'got'
import Promise from 'bluebird'
import config from './config'

const logger = bole('es-client')

const { region, host } = config
const awsConfig = new AWS.Config({ region })

function request (options) {
  const opts = Object.assign({
    host,
    region: awsConfig.region,
    protocol: 'https:',
    headers: {
      'Content-Type': 'application/json'
    },
    json: true
  }, options)
  aws4.sign(opts, awsConfig.credentials)
  const { method, path } = opts
  logger.debug({ method, path, host }, 'Performing request')
  return Promise.resolve(got(opts)).then(resp => resp.body)
}

const es = {}

const METHODS = [
  'get',
  'post',
  'put',
  'delete'
]

METHODS.forEach(method => {
  es[method] = (path, body) => request({
    path,
    method: method.toUpperCase(),
    body: JSON.stringify(body)
  })
})

es.bulk = ops => request({
  path: '/_bulk',
  method: 'POST',
  body: ops.map(op => `${JSON.stringify(op)}\n`).join('')
})

es.getRecord = ({index, type, id}) => es.get(`${index}/${type}/${id}`)
es.indexRecord = ({index, type, id, body}) => es.put(`/${index}/${type}/${id}`, body)
es.deleteRecord = ({index, type, id}) => es.delete(`/${index}/${type}/${id}`)

export default es

The session token must be added after signing.

Was running this repo and this request kept hitting my firewall rules.

54.192.5.251:443
reverse DNS shows this is cloudfront.

Why would this repo need to call out to cloudfront? This raised a ton of red flags with my system as cloudfront is extremely common in malware.

Me again :-)

I am now trying to do POST requests from the browser in a react app and am getting the following error message:

Request URL:https://api.meeshkan.com/v0/userdata
Request Method:POST
Status Code:403 
Remote Address:13.33.244.17:443
Referrer Policy:no-referrer-when-downgrade
Response Headers
content-length:192
content-type:application/json
date:Thu, 23 Nov 2017 19:42:30 GMT
status:403
via:1.1 b5b008573eab794a3818bb6b76b0a164.cloudfront.net (CloudFront)
x-amz-cf-id:OVQoT5AwD9wFeymJBbrS_pDPImTfr5kRutyN6mgSad9ybgMKAT0erw==
x-amzn-errortype:InvalidSignatureException
x-amzn-requestid:713e693b-d086-11e7-95ce-dd22e53cc31d
x-cache:Error from cloudfront
Request Headers
:authority:api.meeshkan.com
:method:POST
:path:/v0/userdata
:scheme:https
accept:application/json, text/plain, */*
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9,fr-FR;q=0.8,fr;q=0.7
authorization:AWS4-HMAC-SHA256 Credential=ASIAIDX6BO2SXYZQYVUA/20171123/eu-west-1/execute-api/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=[signature redacted]
content-length:54
content-type:application/json; charset=UTF-8
origin:http://localhost:8080
referer:http://localhost:8080/
user-agent:Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Mobile Safari/537.36
x-amz-date:20171123T194229Z
x-amz-security-token: [token redacted]
Request Payload
view source
{email: "[email protected]", name: "Cid Caesar"}
email
:
"[email protected]"
name
:
"Cid Caesar"

What's strange is that my react-native app works perfectly, so my guess is that something about issuing this request from the browser makes aws4 behave differently. Not sure what, though... Does anyone spot anything obviously wrong in this construction? It looks pretty clean to my eyes, but maybe I'm too deep down the rabbit hole to notice a glaring issue...

Are you positive SES supports V4 signatures? It's not in the list of supported services.
http://docs.aws.amazon.com/general/latest/gr/sigv4_support.html

Same issue as #46:
TypeError: querystring.unescape is not a function

But in create-react-app it is not possible to configure bundler (webpack) with aliases. I'm currently using https://github.com/nisaacson/aws-v4-sign-small but it would be nice if aws4 would work out of the box too.

While I have reviewed other tickets on the signature for API Gateway calls, I have not been able to sign requests. API Gateway always returns 403 - Invalid Signature.

let request = {
  method: 'GET',
  url: 'https://myapi.execute-api.us-west-2.amazonaws.com/Prod/pets',
  host: 'myapi.execute-api.us-west-2.amazonaws.com',
  baseURL: 'https://myapi.execute-api.us-west-2.amazonaws.com/Prod',
  region: 'us-west-2',
  service: 'execute-api',
  path: '/pets',
  headers: {
    'Content-Type': 'application/json'
  }
}

let signedRequest = aws4.sign(request,
  {
    secretAccessKey: AWS.config.credentials.secretAccessKey,
    accessKeyId: AWS.config.credentials.accessKeyId,
    sessionToken: AWS.config.credentials.sessionToken
  })

delete signedRequest.headers['Host']
let response = await axios(signedRequest)

Any help would be appreciated, perhaps I am missing something simple? I have confirmed that there are values in AWS.config.credentials.

Today I have tried to perform a few requests using a generated secret key like: /zsAn5edKuCsCEwhTyASPuk8RoT5DMukpbcvBB+Z and unfortunately the signatures were failing which then could be fixed when generating a new key without the forward slash in the beginning.

I am not completely sure this was an issue with this library but thought it could be important to mention it here.

Thanks for the great tool.

I see that the body must be string/JSON/text/etc, what if the body is actually a multipart/form-data? How can we pass it on this plugin so it can be calculated as part of signature?

I'm trying to setup headers for API Gateway
I use cognito temporary credentials. I've tried different approaches but I'm still getting InvalidSignatureException

var options = aws4.sign({
            service: 'execute-api',
            region: 'us-east-1',
            hostname: 'https://tzg20cqzbj.execute-api.us-east-1.amazonaws.com',
            path: 'https://tzg20cqzbj.execute-api.us-east-1.amazonaws.com/development/users/',
            headers: {
                'Content-Type': 'application/json',
                'authToken': auth.idToken,
                'Accept': 'application/json'
            }

        }, {
            accessKeyId: AWS.config.credentials.accessKeyId,
            secretAccessKey: AWS.config.credentials.secretAccessKey,
            sessionToken: AWS.config.credentials.sessionToken
        });

        delete options.headers['Host'];

        $http({
            method: 'GET',
            url: 'https://tzg20cqzbj.execute-api.us-east-1.amazonaws.com/development/users/',
            headers: options.headers
        }).then(function successCallback(response) {
            console.log(response);
        }, function errorCallback(response) {
            console.log(response);
        });

Hi,

For some reason when I use the following code in onRequest HTTP Interceptor (angular4) the aws4.sign function it doesn't return a signed object and halts the HTTP request

import * as aws4 from 'aws4';

onRequest(requestOptions: RequestOptionsArgs): RequestOptionsArgs {
console.log('access key:', this.accessKeyId);
console.log('secert:', this.accessSecret);
console.log('session token:', this.sessionToken);

let signedObject = aws4.sign(requestOptions, {
                                   secretAccessKey: this.accessSecret,
                                   accessKeyId: this.accessKeyId,
                                   sessionToken: this.sessionToken,
                            });
console.log('signed object:', signedObject);

return requestOptions;
}

Have confirmed that the access/secret/session tokens all exist but not errors are populated back in the console - I have also tried with aws4.RequestSigner which has the same response (or lack of)

I have also tried the library in a service as the below code;

let request = {
  method: 'GET',
  url:    pageUrl,
  headers: {
    'Content-Type': 'application/json',
    'Accept': 'application/json',
  },
};
let opts = {host: url, path: '?size=' + size + '&cursor=' + cursor};

let signedRequest = aws4.sign(opts, {accessKeyId: this.accessKeyId, secretAccessKey: this.accessSecret, sessionToken: this.sessionToken});
console.log('signedRequets: ', signedRequest);
return this._http.get(pageUrl, {headers: signedRequest});

the console log statement never appears and the app doesn't fire off the HTTP GET request and I cannot figure out why

I have other calls working successfully through the signer which DONT have query params at the end, i have tried to set them in the host + path option, both in the host option (single concat string host + path + params) but nothing appears to work

thanks

How can you consume the results of stdout code (below)? Is it possible to put it into a variable and call a function that on .end()? Thanks!
http.request(opts, function(res) { res.pipe(process.stdout) }).end()

aws4 is broken on Windows because it calls path.normalize on a URL.

Since Windows uses \ instead of / as the path separator, the wrong canonical string is generated and the signature doesn't work.

As mentioned in #16 it might be nice for users of bower to have this packaged up. The main con of this is the extra maintainability overhead of managing another package – not to mention all the browser testing that may or may not need to be done to determine compatibility beyond Node.js.

Having trouble signing a simple GET request using aws4 with axios. I get a CORS error and 403 in the console but digging deeper in the Network tab I see the message "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."

let request = {
  method: 'GET',
  url: 'https://xxxxxxxxxx.execute-api.eu-west-2.amazonaws.com/dev/xxxxxxx',
  path: '/xxxxxxx',
  service: 'execute-api',
  region: 'eu-west-2'
}
let signedRequest = aws4.sign(request, {
  accessKeyId: rootState.auth.accessKeyId,
  secretAccessKey: rootState.auth.secretAccessKey,
  sessionToken: rootState.auth.sessionToken
})
console.log(signedRequest)
delete signedRequest.headers['Host']
axios(signedRequest)
  .then(response => {
    console.log('Status code: ' + response.status)
  })
  .catch(error => {
    return error
  })

I'm able to successfully sign a request to the same endpoint using Postman with its AWS Signature authorization type and using the same tokens so I know that the issue isn't related to the tokens.

Have tried to follow the docs. Am I missing something?

This is not really a problem with this library, but could be fixed here, so I opened an issue here.

If I use "aws4" with browserify, the library throws this error:

TypeError: querystring.unescape is not a function
    at /tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:3245:45
    at Array.map (native)
    at RequestSigner.parsePath (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:3244:28)
    at RequestSigner.prepareRequest (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:3040:8)
    at RequestSigner.sign (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:3090:30)
    at Object.aws4.sign (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:3270:50)
    at Object.module.exports.sign (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:1593:23)
    at Object.module.exports.authorize (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:1555:27)
    at Context.<anonymous> (/tmp/5aec8ddf88b695d277e0f6c174b57e6c.browserify:53837:39)

The issue is that browserify's "querystring" module does not contain the "unescape" method (see mike-spainhower/querystring#4)

Some kind soul is providing a library that actually works: https://github.com/mathiasvr/querystring

Can "aws4" use that implementation? (It would involve depending on the above library, and adding a "browser" directive to the package.json here.)

I don't mind opening a PR, if this change is acceptable :)