UploadPartRequest.StreamTransferProgress Reports Incorrectly about aws-sdk-net HOT 10 CLOSED

Hi, I'll have a look at this and get back to you. Which version of the SDK are you using, and what is the .NET runtime version?
Are you using a local proxy like fiddler which might cache the upload content?

from aws-sdk-net.

No Local Proxies i also tested on a AWS EC2 instance. i tested on both the current release and the release before of the sdk (i cant get exact number right now but if i recall correctly 2.5.1 and 2.6 but i may be mistaken)

from aws-sdk-net.

Thanks for the information. About the SDK version, it'll be great if you can confirm the exact version, the latest release of SDK v1 is 1.5.45 and SDK v2 which supports .NET 4.5 is 2.0.8.2.

from aws-sdk-net.

So i have tested version:
2.0.7.0 .net 3.5 and the second version i install yesterday from the official website download(so i will assume 2.0.8.2) both are the .net 3.5 versions.

from aws-sdk-net.

I am able to see proper event notifications while uploading a part with the following sample code, which uploads a 5 MB file. The network speed for me is much higher though.

Can you share a sample code which replicates the issue.

using Amazon;
using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Model;
using System;
using System.Collections.Generic;
using System.IO;

namespace TransferUtilityProgressTest
{
    class Program
    {
        static long MEG_SIZE = (long)Math.Pow(2, 20);

        static void Main(string[] args)
        {
            var filePath = @"C:\temp\UploadTestFile.txt";
            GenerateFile(filePath, 5 * MEG_SIZE);

            var s3Config = new AmazonS3Config
            {
                RegionEndpoint = RegionEndpoint.USWest2
            };
            var s3Client = new AmazonS3Client(ACCESS_KEY,SECRET_KEY, s3Config);

            var bucketName = BUCKET_NAME;
            var key = "UploadTestFile.txt";
            var initResponse = s3Client.InitiateMultipartUpload(new InitiateMultipartUploadRequest
            {
                BucketName = bucketName,
                Key = key,
                CannedACL = S3CannedACL.BucketOwnerFullControl
            });

            var uploadPartRequest = new UploadPartRequest
            {
                BucketName = bucketName,
                FilePath = filePath,
                Key = key,
                PartNumber = 1,
                UploadId = initResponse.UploadId
            };
            uploadPartRequest.StreamTransferProgress += OnTransferProgress;
            var uploadPartResponse = s3Client.UploadPart(uploadPartRequest);            

            var eTags = new List<PartETag> 
            { 
                new PartETag 
                { 
                    PartNumber = 1, 
                    ETag = uploadPartResponse.ETag 
                }
            };
            s3Client.CompleteMultipartUpload(new CompleteMultipartUploadRequest
            {
                BucketName = bucketName,
                Key = key,
                UploadId = initResponse.UploadId,
                PartETags = eTags,
            });
        }

        private static void OnTransferProgress(object sender, StreamTransferProgressArgs e)
        {
            Console.WriteLine("{0}/{1} {2}%",e.TransferredBytes, e.TotalBytes, e.PercentDone);
        }

        private static void GenerateFile(string path, long size)
        {
            new DirectoryInfo(Path.GetDirectoryName(path)).Create();
            using (StreamWriter stream = new StreamWriter(path))
            {
                for (long i = 0; i < size; i++)
                {
                    char c = (char)('a' + (i % 26));
                    stream.Write(c);
                }
            }
        }
    }
}

from aws-sdk-net.

Yup on much faster connections - such as EC2 1Gbps it very hard to notice the issue as it only seems to happen for the first few megabytes and 5mb i assume you can upload almost instantly? i don't have the sample code with me right now, il have to pass it onto your when i get to work on Monday - do you have a way for me to share with you privately?

from aws-sdk-net.

I tested this code while being connected to a wireless network where it took ~7 seconds for the transfer to complete. The progress updates were in sync with the network activity.

One way to send the sample code would be to private message me on the AWS Developer Forums (https://forums.aws.amazon.com/forum.jspa?forumID=61), my username is milindaws (display name MilindG@AWS).

from aws-sdk-net.

I think accidentally deleted my previous post! so there's 3 differences from the stuff i tested, im doing part uploading in a thread, im using a stream which each part is stored in memory while its being sent and also(for one of the cases) im using WPF.

i changed your code to account for the all of the abovebut currently it does not seem to cause the issue on my home machine - il get back to you on by Monday when i'm using the exact same setup where i see the issue happening.

from aws-sdk-net.

I'm not able to private message you on the forums (don't know how).

see bellow for an example of the problem in action, i'll have to PM you to share code. the interesting thing is that the percent's and bytes line up as well as the total bytes is correct to the file i have picked.

P.S. Sigh... it appears that the antivirus on the machine was causing the issue, as you described with a possible proxy scenario the antivirus takes all the bytes(hence why i saw the behaviour on about 90mb~ but not above)and sends them as it scans.

from aws-sdk-net.

Thanks for the update, glad you got the issue figured out.

from aws-sdk-net.