python lockf and flock behaviour

Jatin Kumar picture Jatin Kumar · Feb 12, 2015 · Viewed 12.4k times · Source

I have read enough posts on stackoverflow regarding the difference b/w flock/lockf/fcntl but I am unable to answer the below observation:

>>> import fcntl
>>> a = open('/tmp/locktest', 'w')
>>> b = open('/tmp/locktest', 'w')
>>> fcntl.lockf(a, fcntl.LOCK_EX | fcntl.LOCK_NB)
>>> fcntl.lockf(a, fcntl.LOCK_EX | fcntl.LOCK_NB)
>>> fcntl.lockf(b, fcntl.LOCK_EX | fcntl.LOCK_NB)
>>>
>>> a.close()
>>> b.close()

>>> a = open('/tmp/locktest', 'w')
>>> b = open('/tmp/locktest', 'w')
>>> fcntl.flock(a, fcntl.LOCK_EX | fcntl.LOCK_NB)
>>> fcntl.flock(a, fcntl.LOCK_EX | fcntl.LOCK_NB)
>>> fcntl.flock(b, fcntl.LOCK_EX | fcntl.LOCK_NB)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 35] Resource temporarily unavailable

Why is the behaviour different in the two cases? I know the obvious answer that these are two different locking mechanisms. I am looking for:

  1. What actually lockf() or flock() does to file (inode/fd)?
  2. As per the demo, are we allowed taking the same lock recursively?

I understand the basics of fds and stuff so I would prefer to have a technical answer with more insights to operating system level details.

OSX 10.9.3, Python: 2.7.5

Answer

pppk520 picture pppk520 · Jan 21, 2016

A nice article about this: On the Brokenness of File Locking

In short:

  • POSIX locks:

    lockf() most of the time implemented as just an interface of fcntl()

    fcntl() locks are bound to processes, not file descriptors. If a process has multiple open file descriptors for a particular file, any one of these file descriptors used for acquiring locking will RESET the lock.

  • BSD lock:

    flock() locks are bound to file descriptors, not processes.

Furthermore

A good analysis with tests: Advisory File Locking – My take on POSIX and BSD locks

Excerpt of Summary:

  • fcntl and flock style locks are completely orthogonal to each other. Any system providing both(Linux does) will be treating locks obtained through each one of them independently.
  • Both POSIX and BSD locks are automatically released when the process exits or is aborted.
  • Both POSIX and BSD locks are preserved across execve calls except when the process has set FD_CLOEXEC flag forcing the file descriptor to be closed, and not inherited by the new process.