rsync incremental file list taking forever

Dale King picture Dale King · Sep 7, 2015 · Viewed 21.6k times · Source

I'm copying from one NAS to another. (Netgear ReadyNAS -> QNAP) i tried Pulling the files by running rsync on the QNAP, and that took forever, so I'm currently trying to push them from the Netgear. The code I'm using is:

rsync -avhr /sauce/folder [email protected]:/dest/folder

i'm seeing:

sending incremental file list

and nothing after that.

File transfer is 577gb and there are a lot of files, however I'm seeing 0 network traffic on the QNAP (It fluctuates between 0kb/s to 6kb/s) so it looks like its not sending any kind of incremental file list.

all folders are created on the destination and then nothing happens after that.

Anyone have any thoughts? Or any ideas on if there is a better way to copy files from a ReadyNAS to QNAP

Answer

I'll Eat My Hat picture I'll Eat My Hat · Oct 9, 2018

The documentation for -v says increase verbosity. If the only thing you're interested in is seeing more progress, you can chain -v together like so:

rsync -avvvhr /sauce/folder/ [email protected]:/dest/folder/

and you should see more interesting progress.

This could tell you if your copying requirements -a are stricter than you need and thus take a lot of unnecessary processing time. For example, I attempted to use -a, which is equivalent to -rlptgoD, on over 100,000 images. Sending the incremental file list did not finish, even overnight. After changing it to

rsync -rtvvv /sauce/folder/ [email protected]:/dest/folder/

sending the incremental file list became much faster, being able to see file transfers within 15 minutes