rsync: --include-from vs. --exclude-from what is the actual difference?

bash unix rsync
Craig picture Craig · Oct 10, 2013 · Viewed 44.9k times · Source

In the documentation, it mentions these as being files containing lists of either patterns to include or patterns to exclude. However, that implies for inclusions, everything is considered an exclusion except where things match patterns. So for example, an include file containing:

/opt/**.cfg

Should only include any file named *.cfg that exists anywhere under a directory named opt any where in the tree. So it would match the following:

/opt/etc/myfile.cfg
/some/dir/opt/myfile.cfg
/notopt/opt/some/other/dir/myfile.cfg

I'd therefore expect it to implicitly exclude anything else. But that doesn't seem to be the case, since I am seeing this in the itemized output:

*deleting   etc/rc.d/init.d/somescript

So what is the deal with --include-from and --exclude-from? Are they just aliases for --filter-from?

Answer

Hari Menon picture Hari Menon · Oct 10, 2013

rsync doesn't work like that. Any file with a filename pattern that does not match any of the include or exclude patterns are considered to be included. In other words, think of the include pattern as a way of overriding exclude pattern.

From the docs (emphasis mine):

Rsync builds an ordered list of include/exclude options as specified on the command line. Rsync checks each file and directory name against each exclude/include pattern in turn. The first matching pattern is acted on. If it is an exclude pattern, then that file is skipped. If it is an include pattern then that filename is not skipped. If no matching include/exclude pattern is found then the filename is not skipped.

So, if you want to include only specific files, you first need to include those specific files, then exclude all other files:

--include="*/" --include="*.cfg" --exclude="*"

Couple of things to note here:

  1. The include patterns have to come before the excludes, because the first pattern that matches is the one that gets considered. If the file name matches the exclude pattern first, it gets excluded.

  2. You need to either include all subdirectories individually, like --include="/opt" --include="/opt/dir1" etc. for all subdirectories, or use --include="*/" to include all directories (not files). I went with the second option for brevity.

It is quirky and not very intuitive. So read the docs carefully (the "EXCLUDE PATTERNS" section in the link) and use the --dry-run or -n option to make sure it is going to do what you think it should do.

Related questions

How can I check if a directory exists in a Bash shell script?

What command can be used to check if a directory exists or not, within a Bash shell script?

Read More
Looping through the content of a file in Bash

How do I iterate through each line of a text file with Bash? With this script: echo "Start!" for p in (peptides.txt) do echo "${p}" done I get this output on the screen: Start! ./runPep.sh: line 3: syntax error …

Read More
How to permanently set $PATH on Linux/Unix?

I'm trying to add a directory to my path so it will always be in my Linux path. I've tried: export PATH=$PATH:/path/to/dir This works, however each time I exit the terminal and start a new terminal …

Read More