How to best capture and log scp output?

logging shell scp
caseyboardman picture caseyboardman · Oct 14, 2008 · Viewed 50.4k times · Source

I am trying to capture output from an install script (that uses scp) and log it. However, I am not getting everything that scp is printing out, namely, the progress bar.

screen output:

Copying /user2/cdb/builds/tmp/uat/myfiles/* to server /users/myfiles as cdb

cdb@server's password: myfile 100% |*****************************| 2503 00:00

log output:

Copying /user2/cdb/builds/tmp/uat/myfiles/* to server /users/myfiles as cdb

I'd really like to know that my file got there. Here's what I am trying now to no avail:

myscript.sh 2>&1 | tee mylogfile.log

Does anyone have a good way to capture scp output and log it?

Thanks.

Answer

Martin picture Martin · Oct 17, 2012

scp prints its progress bar to the terminal using control codes. It will detect if you redirect output and thus omit the progress bar.

You can get around that by tricking scp into thinking it runs in a terminal using the "script" command which is installed on most distros by default:

script -q -c "scp server:/file /tmp/" > /tmp/test.txt

The content of test.txt will be:

file    0%    0     0.0KB/s   --:-- ETA
file   18%   11MB  11.2MB/s   00:04 ETA
file   36%   22MB  11.2MB/s   00:03 ETA
file   54%   34MB  11.2MB/s   00:02 ETA
file   73%   45MB  11.2MB/s   00:01 ETA
file   91%   56MB  11.2MB/s   00:00 ETA
file  100%   61MB  10.2MB/s   00:06

...which is probably what you want.

I stumbled over this problem while redirecting the output of an interactive script into a log file. Not having the results in the log wasn't a problem as you can always evaluate exit codes. But I really wanted the interactive user to see the progress bar. This answer solves both problems.

Related questions

Writing outputs to log file and console

In Unix shell, I have a env file (env file defines the parameters required for running the user script like log file name and path, redirect outputs and errors to log file, database connection details, etc) which redirects all the …

Read More
live output from subprocess command

I'm using a python script as a driver for a hydrodynamics code. When it comes time to run the simulation, I use subprocess.Popen to run the code, collect the output from stdout and stderr into a subprocess.PIPE --- …

Read More
redirect COPY of stdout to log file from within bash script itself

I know how to redirect stdout to a file: exec > foo.log echo test this will put the 'test' into the foo.log file. Now I want to redirect the output into the log file AND keep it on …

Read More