Python: best way to catch env variables from parsed arguments?

Question

Let say we have a script that accepts arguments, which then dispatches it to subprocess run. When there are no ENV defined, I can simply dispatch all args directly and call commands, like:

import subprocess
import sys

subprocess.run(sys.argv[1:], check=True)

But if I let say call like this my-script.py A=22; B=33; echo "test", then first argument is found as environment variable, not executable.

Is there some good way to recognize when we have ENV before command to handle it?

P.S. I could use Popen with shell=True to propagate it to shell, but I want to preprocess command before executing it (like do extra stuff if I got specific ENV passed etc).

I don't really see any environment variables in the call you show. (`bash`, at least, has an option to allow any assignment-like word in a command to be treated as an environment modifier, so *maybe* `A=22` sets an environment variable, but `B=33` and `echo "test"` are just two shell commands unrelated to your Python script.) — chepner, Dec 03 '21 at 16:17
It sounds like you are referring to an `ENV` directive in a Dockerfile. — chepner, Dec 03 '21 at 16:18
No, Im talking about like this `https://stackoverflow.com/questions/10856129/setting-an-environment-variable-before-a-command-in-bash-is-not-working-for-the` — Andrius, Dec 03 '21 at 16:21
It's interesting that `my-script.py A=22; B=33; echo "test"` results in sys.argv ['my-script.py', 'A=22']. I'm thinking that Python is built off of C and C puts special significance in `;` that it immediately stops upon reaching a `;`. When you run `my-script.py A=22 B=33 echo "test"` without the `;`'s the sys.argv will list all of your arguments. — Ryan Nygard, Dec 03 '21 at 16:29
This has nothing to do with C or Python, but with the shell. `;` is a command terminator. — chepner, Dec 03 '21 at 16:44

score 2 · Accepted Answer · answered Dec 03 '21 at 16:19

2

It depends on the shell you use.

When you write:

my-script.py A=22; B=33; echo "test"

It is 3 commands to (for example Bash) shell:

my-script.py A=22
B=33
echo "test"

What you should do, if you want to pass env vars to your script, is:

export B=33
my-script.py A=22
echo "test"

Then your script will receive argument "A=22", and env var B will be available to check in it.

answered Dec 03 '21 at 16:19

Gnudiff

4,297
1
24
25

Oh right, I could just use `export MYVAR=something` as separate command. That makes sense. – Andrius Dec 03 '21 at 16:23
Putting the assignment *before* the command also sets the environment variable, but only in the environment of Python, not the shell that runs Python , for example `B=33 my-script.py A=22`. If you run `set -k` in the shell first, then `A=22` will *also* be parsed as an environment assignment, equivalent to `A=22 B=33 my-script.py`, rather than passing the string `A=22` as an argument to your script. – chepner Dec 03 '21 at 16:46
@Andrius they *are* separate commands even in your original post. Semicolon serves as command separator in Bash. Your python script never *receives* anything past the ";" sign the way you wrote it. – Gnudiff Dec 03 '21 at 17:54
I know. I can treat it as separate command. Thats no problem – Andrius Dec 03 '21 at 18:13

score 1 · Answer 2 · answered Dec 03 '21 at 16:38

The simplest way would be to use argparse, setup optional arguments for the ENV variabled and consider all positional arguments as the command to be executed. Example:

import argparse
import subprocess

argp = argparse.ArgumentParser()
argp.add_argument(
    "-e", "--envvar",
    help="Add ENVVAR to the environment of the executable to be run (may be specified multiple times",
    nargs=1, # one ENVVAR per occurrence …
    action="append" # … which are all gathered into a list
)

argp.add_argument(
    "executable",
    help="Shell script or path to executable",
    nargs="*" # Take all remaining positional arguments for this option
)

args = argp.parse_args()

for envvar in args.envvar:
    print("ENVVAR:", envvar)

subprocess.run(args.envvar + args.executable, shell=True)

If you don't want that, you could use a regex to search for arguments of the right form:

import re
import sys

envvars = []
executable = []
for arg in sys.argv[1:]:
    if re.match("(a-zA-Z0-9)=.*;", arg):
        envvars.append(arg.rstrip(";"))
    else:
        executable.append(arg)

subprocess.run(envvars + executable, shell=True)

Python: best way to catch env variables from parsed arguments?

2 Answers2