Comment by Dunedan

Comment by Dunedan 10 hours ago

3 replies

I wonder if the at least partially the reason for the speed up isn't the multi-threading, but instead that rclone maybe doesn't compress transferred data by default. That's what rsync does when using SSH, so for already compressed data (like videos for example) disabling SSH compression when invoking rsync speeds it up significantly:

  rsync -e "ssh -o Compression=no" ...
dspillett 5 hours ago

IIRC rsync uses your default SSH options, so turning off compression is only needed if your default config explicitly turns it on (generally or just for that host). If sending compressible content using rsync's compression instead of SSH's is more effective when updating files because even if not sending everything it can use it to form the compression dictionary window for what does get sent (though for sending whoe files, SSH's compression may be preferable as rsync is single threaded and using SSH's compression moves that chunk of work to the SSH process).

nh2 10 hours ago

Compression is off by default in OpenSSH, at least `man 5 ssh_config` says:

> Specifies whether to use compression. The argument must be yes or no (the default).

So I'm surprised you see speedups with your invocation.

  • Dunedan 8 hours ago

    Good point. Seems like I enabled it in ~/.ssh/config ages ago and did forget about it. Nonetheless, it's good to check whether it's enabled when using rsync to transfer large, already well compressed files.