SFTP file lock mechanism

locking sftp transfer
CyberDracula picture CyberDracula · Nov 14, 2014 · Viewed 15.1k times · Source

How can I make sure that a file uploaded through SFTP (in a Linux base system) stays locked during the transfer so an automated system will not read it?

Is there an option on the client side? Or server side?

Answer

Martin Prikryl picture Martin Prikryl · Nov 14, 2014

SFTP protocol supports locking since version 5. See the specification.

You didn't specify, what SFTP server are you using. So I'm assuming the most widespread one, the OpenSSH. The OpenSSH supports SFTP version 3 only, so it does not support locking.

Anyway, even if your server supported file locking, most SFTP clients/libraries won't support SFTP version 5. Or even if they do, they won't support the locking feature. Note that the lock is explicit, the client has to request it.

There are some common workarounds for the problem:

  • As suggested by @user1717259, you can have the client upload a "done" file, once an upload finishes. Make your automated system wait for the "done" file to appear.

  • You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.

  • Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after an upload to its final name. Make your automated system ignore the ".filepart" files.

    See (my) article Locking files while uploading / Upload to temporary file name for example of implementing this approach.

    Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive (courtesy of @fakedad).

  • A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.

  • You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, when you download an incomplete file.

Related questions

Find out who is locking a file on a network share

I want to known who is locking a file on a network share. Here is the problem : the network share is on a NAS, so I can't log on. I need a tool to find out remotely who is locking …

Read More
How does lock work exactly?

I see that for using objects which are not thread safe we wrap the code with a lock like this: private static readonly Object obj = new Object(); lock (obj) { // thread unsafe code } So what happens when multiple threads access the …

Read More
Optimistic vs. Pessimistic locking

I understand the differences between optimistic and pessimistic locking. Now could someone explain to me when I would use either one in general? And does the answer to this question change depending on whether or not I'm using a stored …

Read More