rsync skip non existing files on source
I'm developing a script in python that could collect logs from multiple machines. I'm using rsync. But there's a problem. I have logs for multiple services that look like:
service1.log
service2.log
service3.log
... and so on
Paths to this files and folders are specified in code. But sometimes I get in situation when some log files do not exist yet. And rsync does not finish successfully.
How can I skip files that do not exist on source machine?
P.S. I'm using saltstack to rule machines, so I call:
__salt__['cmd.retcode']('rsync -a file1 file2...')
Answer
Use --ignore-missing-args:
The version 3.1.0 man page says,
--ignore-missing-args
When rsync is first processing the explicitly requested source files (e.g. command-line arguments or --files-from entries), it is normally an error if the file cannot be found. This option suppresses that error, and does not try to transfer the file. This does not affect subsequent vanished-file errors if a file was initially found to be present and later is no longer there.
For example:
% rsync --version
rsync version 3.1.0 protocol version 31
% rsync bogusfile dest
rsync: link_stat "/tmp/bogusfile" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.0]
% rsync --ignore-missing-args bogusfile dest
# <no error>
The --ignore-missing-args option was added some time between version 3.06 and 3.1.0.
The online version 3.06 rsync man page does not mention it. But the git repository indicates this option was added in 2009.