3

I want create a symlink, overwriting an existing file or symlink if needed.

I've discovered that os.path.exists only returns True for non-broken symlinks, so I'm guessing that any test must also include os.path.lexists.

What is the most atomic way to implement ln -sf in python? (Ie, preventing a file being created by another process between deletion and symlink creation)

Differentiation: This question doesn't specify the atomic requirement

Tom Hale
  • 40,825
  • 36
  • 187
  • 242
  • If you prepare `ln -s file tmplink`, then `mv tmplink link` is atomic. – Amadan Apr 18 '19 at 06:49
  • @Amadan thanks for the suggestion. I still see a security hole, but I hope I got it as good as possible in [my answer](https://stackoverflow.com/a/55741590/5353461). – Tom Hale Apr 18 '19 at 08:13
  • FWIW, `ln -sf` itself **is not actually atomic**. GNU Coreutils [internally implements](https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/force-link.c?h=v8.32#n143) the solution that @Amadan proposed (and that the currently-accepted answer implements); FreeBSD and Busybox simply *delete the destination file* before linking. So "par" is actually pretty easy to clear for this. – JamesTheAwesomeDude Jun 23 '21 at 14:46

1 Answers1

1

This code tries to minimise the possibilities for race conditions:

import os
import tempfile

def symlink_force(target, link_name):
    '''
    Create a symbolic link link_name pointing to target.
    Overwrites link_name if it exists.
    '''

    # os.replace() may fail if files are on different filesystems
    link_dir = os.path.dirname(link_name)

    while True:
        temp_link_name = tempfile.mktemp(dir=link_dir)
        try:
            os.symlink(target, temp_link_name)
            break
        except FileExistsError:
            pass
    try:
        os.replace(temp_link_name, link_name)
    except OSError:  # e.g. permission denied
        os.remove(temp_link_name)
        raise

Note:

  1. If the function is interrupted (e.g. computer crashes), an additional random link to the target might exist.

  2. An unlikely race condition still remains: the symlink created at the randomly-named temp_link_name could be modified by another process before replacing link_name.

I raised a python issue to highlight the issues of os.symlink() requiring the target not exist.

Credit to Robert Seimer's input.

Tom Hale
  • 40,825
  • 36
  • 187
  • 242
  • I raised an [issue](https://bugs.python.org/issue36656) for the security hole mentioned above. – Tom Hale Apr 18 '19 at 08:50
  • You could handle the (unlikely) race by putting the `mktemp`+`symlink` calls inside a loop, retrying until you win the race. Obviously you would need to check that `symlink` failed because of `EEXIST` and not for some other reason that's never going to let it succeed. And for efficiency you'd probably want to hoist the embedded `dirname` call above the loop. – ottomeister Apr 18 '19 at 21:17
  • @ottomeister Thanks, your suggestion helps. But how to prevent the `mktemp`-named file being changed after creation and before replacing the symlink? – Tom Hale Apr 19 '19 at 05:40
  • You can't prevent that. Any badly-behaved process that has write access to this directory can destroy the whole arrangement. It's just as possible that something could replace or delete the new symlink _after_ the rename, as it is that something could replace or delete it _before_ the rename. But if you're worried about concurrent `mktemp`s accidentally colliding on the same name then you could `mkdtemp` a directory, create the new symlink in that directory, then `rename` the symlink over the original one (and then `rmdir` the temp directory). A `rename` within a filesystem is atomic. – ottomeister Apr 19 '19 at 07:55
  • I am somewhat astounded that this didn't come OOTB. – JamesTheAwesomeDude Feb 02 '21 at 00:14