Can't get simple putStream() script to work... about knox HOT 12 CLOSED

You need to send Content-Length or use putFile to get it automatically. Without Content-Length Node automatically sets some weird header (Transfer-Encoding: chunked, maybe?) which Amazon rejects.

from knox.

@domenic let's put a warning in there if Content-Length is not set

from knox.

Unfortunately we need to take it upon ourselves to improve the error reporting at the client layer, because Amazon does a terrible job with that.

from knox.

We can explicitly disable chunked encoding in node by forcing Connection: close. Getting the "Content-Length" in my case would require Buffering a HTTP request in memory, which I want to avoid.

from knox.

So in the absence of Content-Length in the request we should do that then.

from knox.

If S3 doesn't support that though, we probably just need to stat the file @TooTallNate and send it along as Content-Length

from knox.

Forcing "identity" transfer encoding makes the 501 turn into a 403, but it still doesn't work...

var headers = {};
headers['content-type'] = 'image/jpeg';
headers['transfer-encoding'] = 'identity';
headers['connection'] = 'close';

Gives back XML like:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>SignatureDoesNotMatch</Code>
  <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
  <StringToSignBytes>50 55 54 0a 0a 69 6d 61 67 65 2f 6a 70 65 67 0a 46 72 69 2c 20 32 31 20 53 65 70 20 32 30 31 32 20 32 31 3a 35 31 3a 31 32 20 47 4d 54 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 75 62 6c 69 63 2d 72 65 61 64 0a 2f 63 6c 6f 75 64 75 70 2d 66 69 6c 60 73 2f 6e 61 74 65 2d 61 76 61 74 61 72 2e 6a 70 67</StringToSignBytes>
  <RequestId>D66D3C6DBC680FFF</RequestId>
  <HostId>1L58FhSdqqis1t5xH87Z5cBS/Rvwn2IVXq6CLa/Xa7E3hSkEAqbEw+AtnmX0e/xy</HostId>
  <SignatureProvided>cHw1NkjS/vzwuU6haCE+Rzj2mMI=</SignatureProvided>
<StringToSign>PUT

image/jpeg
Fri, 21 Sep 2012 21:51:12 GMT
x-amz-acl:public-read
/cloudup-files/nate-avatar.jpg</StringToSign>
  <AWSAccessKeyId>...</AWSAccessKeyId>
</Error>

from knox.

Notably stat'ing the file to get its Content-Length is exactly what putFile does.

from knox.

Oh interesting, I wonder if this uncovers a deficiency in the signing algorithm or if Amazon is just giving inaccurate error messages. Hmm!

from knox.

It would be killer if knox supported a multipart upload handling API (or did this behind the scenes!) http://stackoverflow.com/questions/8653146/can-i-stream-a-file-upload-to-s3-without-a-content-length-header

from knox.

@TooTallNate pull request welcome!!

from knox.

Added early error in 41cc7a6 which will be released shortly as 0.3.1.

from knox.