OAuth timeout after 1 hour of pushing. about drive HOT 15 CLOSED

Hey Joel, thank you for the information related to the issue. Given the error description and the time taken, this seems to me like the OAuth token expired/needed refreshing. In deed wav file conversion seems to take ages. So a quick guess is that the conversion is the bottleneck.
A push of a 367.34KB wav file without performing conversion takes 4s, with -convert passed in takes 22s.
In relation to the title of the issue, am able to upload two mp4 files (1.93 GB in total) in about 22 minutes. Quick question would you mind uploading larger files of a different format and also not setting flag -convert?

from drive.

Also btw would you mind checking your quota to see if you still have space? This could be the actual cause of the invalid credentials error.

$ drive quota

from drive.

Gah! I forgot to tell you what command line options I was specifying above. Sorry.

Here is this evenings attempt. It seems to be mostly confined to those wav files now:

I'm not really after any kind of conversion for any of my files. I wondering if the wav file conversion is happening even though I'd specified false ... ???

Quota details

[darkvortex@infuser gdrive]$ drive quota
Name: Joel Sutton
Account type:   LIMITED
Bytes Used:     20622364852          (19.21GB)
Bytes Free:     86751817548          (80.79GB)
Bytes InTrash:  371787057            (354.56MB)
Total Bytes:    107374182400         (100.00GB)

* Space used by Google Services *
Service                              Bytes                               
DRIVE                                19.21GB                             
GMAIL                                2.90GB                              
PHOTOS                               79.30MB                             
Space used by all Google Apps        22.18GB                             

drive version 0.0.7

As far as versions go I did checkout with git checkout fix/mkdirAll-feature when compiling as outlined in issue #51

Definitely happy to upload a large no-wav format file. I'll try uploading a CentOS iso file tonight.

Thanks :)

from drive.

I just briefly got up but would you mind checking out master and try with the code there.

$ git checkout master
$ go get -u github.com/odeke-em/drive/cmd/drive
$ drive push -ignore-checksum

Once that has been tried out and if persists or even still, our next experiment would be for you to try uploading any one of the failing files via the web UI and timing it.
Thanks for the patience and for eliminating out the possibilities.

from drive.

Any progress on checking if this issue still persists? Please pull from master.

from drive.

Apologies for the delay in getting back to you on this.

I wasn't quite able to resolve the problem with the wav files.

However I was able to transfer most of the 50 gb of data from my backup linux vps into Google Drive before the renewal fee was due. The few items that had problems were picked up by the official google drive app running on my Macintosh, and uploaded.

That's saved me a huge amount of transfer time on my slow internet link here at home. So a big thank you to you and any other contributors :) :) :)

I've just installed go and drive on my mac now so I'll see if I can replicate the problem with a large wav file. I'll also try an iso file to see if it acts any differently.

from drive.

I ran drive this morning with an ISO file (see the pic for the command line options). I am on the master branch. Note the error message and, of course, no new file in google drive.

After re-reading the ticket I've realised that you wanted me to run with -ignore-checksum so I'll do that again tonight. Sorry. My Bad.

from drive.

Same issue when ignoring checksums. So it looks like it's not just wav files.

The iso file is 649 mb FYI.

from drive.

I wouldn't know how to reproduce that since as I've previously mentioned I have been able to push 1.93GB in about 23 minutes and I can see that was 649MB in almost 3 hours. I suspect that is an error to do with the refresh token expiring. I'll investigate a bit more. Thank you for the patience and for the continued investigation.

from drive.

I had a similar issue pushing a 1.35GB .mp4 file. However, using -ignore-checksum worked and I uploaded 1.35GB in 53 minutes.

from drive.

I played with bandwidth limitation, like this

$ pv --rate-limit 500k file | drive push -piped file

If controlling the bandwidth limit carefully to make the upload time exceed 60 minutes, I will get the same error.

My example:
0:59:50 works

$ pv --rate-limit 535k 201301010318.tar.gz | drive push -piped 201301010318.tar.gz
1.83GiB 0:59:50 [ 535KiB/s] [==========================================================================>] 100%

and 1:03:24 failed.

$ pv --rate-limit 535k 201304020321.tar.gz | drive push -piped 201304020321.tar.gz
1.94GiB 1:03:24 [ 534KiB/s] [==========================================================================>] 100%            
201304020321.tar.gz: googleapi: Error 401: Invalid Credentials, authError
googleapi: Error 401: Invalid Credentials, authError

and this is reproducible.

from drive.

Thank you very much @kcwu for the investigation and reproduction. In deed, this confirms my initial thoughts about token expiration. Also it gives an exact time frame, and with this we have the tools needed to try to solve this.

from drive.

Closing this issue in favour of moving it to #116

from drive.

Please get the latest from master, it should be fixed.

from drive.

Hey folks please get the latest from master by

$ go get -u github.com/odeke-em/drive/drive-gen && drive-gen

Uploading should be faster and there is also a mechanism for exponential backoff on encountering retryable errors.

from drive.