Improperly set "Range" header when retrying file download from Dropbox about rclone HOT 3 CLOSED

Thanks for reporting this, it does look like a bug. And thank you for your excellent log and analysis.

The problem seems to be in the re-open code

First failure

Reopening on read failure after offset 20766346465 bytes: retry 1/10: unexpected EOF

Reopens with range request - looks good

Range: bytes=20766346465-72200451298

Second failure

Reopening on read failure after offset 21662613633 bytes: retry 2/10: ... i/o timeout

Reopens with incorrect range request - should clearly start from 21662613633 as per the log

Range: bytes=20766346465-72200451298

Third failure

Reopening on read failure after offset 34544399248 bytes: retry 3/10: ... i/o timeout

Re-opens with the same incorrect range request

Range: bytes=20766346465-72200451298

I can reproduce this by patching the dropbox backend to return errors every 1000 bytes.

I've managed to fix this I think.

It turned out to be quite a subtle bug and I think it has caused a lot of issues with rclone downloading too much data over the years.

Please give this a go

v1.67.0-beta.7869.a97c790b8.fix-7759-dropbox-retry on branch fix-7759-dropbox-retry (uploaded in 15-30 mins)

from rclone.

I can confirm the bug is fixed: rclone now requests the correct ranges.
I also checked the hash on the Backblaze via the unpatched release of rclone and it matched with the hash that Dropbox has.

Thank you very much for the quick fix. 👍

from rclone.

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.67 and I'll put it in the upcoming v1.66.1 also.

Thanks for tracking this down - I think this has been a problem in rclone for some time and I'm very glad to fix it.

from rclone.