Verify LFTP transfer success

Question

I am using lftp to transfer files from local to a remote server which only allows SFTP access. I am using the following syntax to transfer the files :

lftp -c "open -u $UN,$Pass sftp://$Server ; mirror -R $Directory"

And this is working all fine when it comes to transffering file. Since I am using this as a cron embedded in a .sh file and sometime the lftp fails (for unknown reason).

How can I verify the transfer using some exit code or something which can be used to ensure the correctness of tranfer?

Answer 1

You'll have to wrap it inside a shellscript, I think.

The shellscript must generate an lftp script like, for instance:

set xfer:log true
set xfer:log-file "/path/to/somewhere/$DATESTAMP.log"
lcd "/path/of/source"
open -u $UN,$Pass sftp://$Server
mirror -R $Directory
quit

Execute by lftp -f $ScriptName.

Afterwards, parse the generated logfile; AFAIK it contains only successful transfers.

awk '{print $3,$6}' $DATESTAMP.log | sed -r -e 's/ [0-9]+-/ /'

The above will generate a list of files and bytes transferred for each file. You can loop over the result and compare with actual file size, for example

# WARNING: Script assumes that filename contains no space.
while read fname fsize; do
  # The following line used if the lftp runs from e.g. the source folder
  actualname=$Prefix/$fname
  actualsize=$(stat -c "%s" "$actualname")
  if (( actualsize != fsize )); then
    # Some error handling code here
  fi
done <<< "$(awk '{print $3,$6}' $DATESTAMP.log | sed -r -e 's/ [0-9]+-/ /')"

Note: I am using a "herestring" there instead of piping, to enable manipulation of variables within the while loop.