Python: module for creating PID-based lockfile?

Question

I'm writing a Python script that may or may not (depending on a bunch of things) run for a long time, and I'd like to make sure that multiple instances (started via cron) don't step on each others toes. The logical way to do this seems to be a PID-based lockfile… But I don't want to re-invent the wheel if there is already code to do this.

So, is there a Python module out there which will manage the details of a PID-based lockfile?

Answer 1

This might be of help to you: lockfile

Answer 2

If you can use GPLv2, Mercurial has a module for that:

http://bitbucket.org/mirror/mercurial/src/tip/mercurial/lock.py

Example usage:

from mercurial import error, lock

try:
    l = lock.lock("/path/to/lock", timeout=600) # wait at most 10 minutes
    # do something
except error.LockHeld:
     # couldn't take the lock
else:
    l.release()

Answer 3

i've been pretty unhappy with all of those, so i wrote this:

class Pidfile():
    def __init__(self, path, log=sys.stdout.write, warn=sys.stderr.write):
        self.pidfile = path
        self.log = log
        self.warn = warn

    def __enter__(self):
        try:
            self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL)
            self.log('locked pidfile %s' % self.pidfile)
        except OSError as e:
            if e.errno == errno.EEXIST:
                pid = self._check()
                if pid:
                    self.pidfd = None
                    raise ProcessRunningException('process already running in %s as pid %s' % (self.pidfile, pid));
                else:
                    os.remove(self.pidfile)
                    self.warn('removed staled lockfile %s' % (self.pidfile))
                    self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL)
            else:
                raise

        os.write(self.pidfd, str(os.getpid()))
        os.close(self.pidfd)
        return self

    def __exit__(self, t, e, tb):
        # return false to raise, true to pass
        if t is None:
            # normal condition, no exception
            self._remove()
            return True
        elif t is PidfileProcessRunningException:
            # do not remove the other process lockfile
            return False
        else:
            # other exception
            if self.pidfd:
                # this was our lockfile, removing
                self._remove()
            return False

    def _remove(self):
        self.log('removed pidfile %s' % self.pidfile)
        os.remove(self.pidfile)

    def _check(self):
        """check if a process is still running

the process id is expected to be in pidfile, which should exist.

if it is still running, returns the pid, if not, return False."""
        with open(self.pidfile, 'r') as f:
            try:
                pidstr = f.read()
                pid = int(pidstr)
            except ValueError:
                # not an integer
                self.log("not an integer: %s" % pidstr)
                return False
            try:
                os.kill(pid, 0)
            except OSError:
                self.log("can't deliver signal to %s" % pid)
                return False
            else:
                return pid

class ProcessRunningException(BaseException):
    pass

to be used something like this:

try:
    with Pidfile(args.pidfile):
        process(args)
except ProcessRunningException:
    print "the pid file is in use, oops."

Answer 4

I know this is an old thread, but I also created a simple lock which only relies on python native libraries:

import fcntl
import errno


class FileLock:
    def __init__(self, filename=None):
        self.filename = os.path.expanduser('~') + '/LOCK_FILE' if filename is None else filename
        self.lock_file = open(self.filename, 'w+')

    def unlock(self):
        fcntl.flock(self.lock_file, fcntl.LOCK_UN)

    def lock(self, maximum_wait=300):
        waited = 0
        while True:
            try:
                fcntl.flock(self.lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB)
                return True
            except IOError as e:
                if e.errno != errno.EAGAIN:
                    raise e
                else:
                    time.sleep(1)
                    waited += 1
                    if waited >= maximum_wait:
                        return False

Answer 5

I believe you will find the necessary information here. The page in question refers to a package for building daemons in python: this process involves creating a PID lockfile.

Answer 6

There is a recipe on ActiveState on creating lockfiles.

To generate the filename you can use os.getpid() to get the PID.

Answer 7

You can try PID: https://pypi.org/project/pid/

As the documentation shows, you can lock a function simply adding the decorator @pidfile() on the top of function/method name.

from pid.decorator import pidfile


@pidfile()
def main():
  pass

if __name__ == "__main__":
  main()

The default location for pidfile self check (the file who says if you can execute the code or not) is '/var/run'. You can change it as follows:

@pidfile(piddir='/path/to/a/custom/location')

For other params, see: https://github.com/trbs/pid/blob/95499b30e8ec4a473c0e6b407c03ce644f61c643/pid/base.py#L41

Unfortunatly, this lib's documentation is a little bit poor.