How to call LC_ALL=C sort from python subprocess

Question

How to call LC_ALL=C sort -k1 file -o file with python subprocess?

When I try: subprocess.check_call(["LC_ALL=C sort -k1 {} -o {}".format(file,file)])

Following error occurs: FileNotFoundError: [Errno 2] No such file or directory: 'LC_ALL=C sort -k1 file.txt -o file.txt'

See also https://stackoverflow.com/questions/2231227/python-subprocess-popen-with-a-modified-environment — larsks, Jul 11 '18 at 20:13

tripleee · Answer 1 · 2018-07-11T20:17:48.170

4

The syntax var=value command is shell syntax to set an environment variable and run a command. subprocess by default does not offer a shell (and though you can get one with shell=True you should generally try to avoid this if you can). The way to do this in Python is to pass in a dictionary of variables with env.

myenv = os.environ.copy()
myenv['LC_ALL'] = 'C'
subprocess.check_call(['sort', '-k1', file, '-o', file], env=myenv)

Python is perfectly capable of sorting a sequence of lines internally; avoiding the external process entirely is an even better solution.

edited Jul 11 '18 at 20:17

answered Jul 11 '18 at 20:12

tripleee

175,061
34
275
318

Thanks for pointing this out; edited now to copy os.environ – tripleee Jul 11 '18 at 20:16
That's a cleaner approach than the one I used; might adopt it. :) – Charles Duffy Jul 11 '18 at 20:17

Charles Duffy · Answer 2 · 2018-07-11T20:33:39.343

Assignments are shell syntax.

When you don't set shell=True, there is no shell, so there's nothing available to parse and honor the assignment.

Just setting `shell=True` would cause security bugs.

If your filename contains $(rm -rf ~), using ''.format() to inject it is dangerous.

So generate an environment that has `LC_ALL` added to it, with a value of `C`:

sort_env = os.environ.copy()
sort_env['LC_ALL'] = 'C'

subprocess.check_call(['sort', '-k1', file, '-o', file], env=sort_env)

Or use `shell=True`, but pass arguments out-of-band:

When a list is passed to shell=True, the first element is treated as the script to interpret; the second as $0 in the context of that script; the third as $1, etc. Thus:

subprocess.check_call(['LC_ALL=C sort -k1 "$1" -o "$2"', '_', file, file])

score 0 · Answer 3 · answered Jul 11 '18 at 20:16

0

Pass a dict of the environmental variables as the env kwarg to the subprocess invocation.

    Python 2.7.14 (default, Mar 22 2018, 14:43:05)

    In [1]: import subprocess

    In [2]: subprocess.check_call(["env"], env={'LC_ALL': 'C'})
    LC_ALL=C

answered Jul 11 '18 at 20:16

mateor

1,293
1
16
19

How to call LC_ALL=C sort from python subprocess

3 Answers3

Assignments are shell syntax.

Just setting shell=True would cause security bugs.

So generate an environment that has LC_ALL added to it, with a value of C:

Or use shell=True, but pass arguments out-of-band:

Just setting `shell=True` would cause security bugs.

So generate an environment that has `LC_ALL` added to it, with a value of `C`:

Or use `shell=True`, but pass arguments out-of-band: