Comment by plasticsoprano

Comment by plasticsoprano 13 hours ago

4 replies

How do you deal with how poorly rclone handles rate limits? It doesn't honor dropbox's retry-after header and just adds an exponential back off that, in my migrations, has resulted in a pause of days.

I've adjusted threads and the various other controls rclone offers but I still feel like I'm not see it's true potential because the second it hits a rate limit I can all but guarantee that job will have to be restarted with new settings.

darthShadow 11 hours ago

> doesn't honor dropbox's retry-after header

That hasn't been true for more than 8 years now.

Source: https://github.com/rclone/rclone/blob/9abf9d38c0b80094302281...

And the PR adding it: https://github.com/rclone/rclone/pull/2622

  • plasticsoprano 3 hours ago

    Interesting. I’ve been through 4 large transfers to Dropbox in the last 3 years and never once has it honored that header.

coreylane 12 hours ago

I honestly haven't used it with Dropbox before, have you tried adjusting --tpslimit 12 --tpslimit-burst 0 flags? Are you creating a dedicated api key for the transfer? Rate limits may vary between Plus/Advanced forum.rclone.org is quite active you may want to post more details there.

  • plasticsoprano 3 hours ago

    I have made a dedicated api application and I have adjusted the tps flags. I’ve scanned the forums a few times but I’ve yet to inquire there.