I've some existing scripts wherein am using ftp + .netrc.
I want to switch to sftp now but it seems it doesn't support macros / .netrc.
Is there any other alternative?
Please help.
Answer
Simply put, you cannot use .netrc with sftp, scp or ssh. These products are part of the OpenSSH standard, which has the keyword 'secure' in the name. It is not a secure practice to automate logins the way .netrc does, and the standard prohibits this kind of automation (storing passwords). There is definitely an alternative, three actually.
Authorization
For either of the first two alternatives, you will want to setup keys and exchange them. On the machine you are connecting from run ssh-keygen, for your purposes it will be much simpler if you do not give the key a pass-phrase, though this is risky. You now have two files in .ssh/, an id_rsa and a id_rsa.pub. Of these the id_rsa must be kept secret or secured (hence the pass-phrase). The pub file is actually one line of text. This one line can be added to the ~/.ssh/authorized_keys file on the receiving host's side. You can add the key to the file manually; but there is also ssh-copy-id shortcut command which does just that, also taking care of file permissions. Having authorized a key, you should be able to connect from the machine with the private key to the machine which has the authorized public key, when you connect as the appropriate user. Test it with ssh -v. If you entered a pass-phrase, you will be prompted for it; if you did not you are now automation ready. You can use an ssh-agent to keep a private key active between sessions while only entering the pass-phrase once. If you are making multiple ssh hops, the option to forward agents will allow the private key from the original sourced box's ssh-agent to be communicated though each hop. Personally I find this overwrought, and hence suggest not using a pass-phrase.
Now that you can make ssh, sftp, and scp connections without entering any password or pass-phrase you're ready to automate the rest.
Alternative 1,
is the preferred alternative were you convert your .netrc macro to a shell script or other script calling a few scp commands. This is similar to automating all your ftp connections with curl or wget. E.G.:
scp -qr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $LOCAL_PATH_FILE_OR_DIR #download
scp -qr $LOCAL_PATH_FILE_OR_DIR $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR #upload
scp -pqr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $USER@$REMOTE_HOST2:$PATH_FILE_OR_DIR #mirror between separate hosts.
ssh $USER@$REMOTE_HOST chmod 644 $PATH_FILE #set permissions
Alternative 2,
using sftp as you mentioned, you can script it with the expects command, with a batch file using the -b option, or by piping commands into sftp. This is a little more similar to an .netrc macro, but has no advantage over alternative 1. I'll show an example of the latter:
#!/bin/sh
echo "OK, starting now..."
sftp -b /dev/fd/0 remotehost <<EOF
cd pub
ascii
get filename.txt
bye
EOF
Alternative 3,
use an sftp program that breaks the SSH standard by allowing you to store connection parameters such as the password. For example using cyberduck and AppleScript, or FileZilla and a queue.
Further notes:
There is an ~/.ssh/config file you can use to give hostnames shorter names, set forwarding parameters, default directories, default usernames, and specific identities for each host. I also like the -l option of scp which limits my transfer rate to something more reasonable.
P.S. You'd think there's a tool out there for converting .netrc macros to (alternative 1 styled) shell scripts. But I found nothing. Is that a tiny niche business opportunity?