3

I would like to write a Python script that takes some necessary positional and some optional command-line arguments via argparse:

  • Let's call the positional args a,b,c, and the optional arguments x,y,z.
  • In my Python script, I would like to pass these args on to a function; specifically, I want a,b,c to be passed as *args, and x,y,z to be passed as **kwargs, the latter retaining their names.
  • I would like to do this many times with different functions and different numbers of positional and optional arguments. Is there a neat, flexible, and/or pythonic way to do this?

Here is some example code:

import argparse

def parse():
    parser = argparse.ArgumentParser()
    parser.add_argument('a', help='1st arg')
    parser.add_argument('b', help='2nd arg')
    parser.add_argument('c', help='3rd arg')
    parser.add_argument('-x', '--x', help='1st kwarg')
    parser.add_argument('-y', '--y', help='2nd kwarg')
    parser.add_argument('-z', '--z', help='3rd kwarg')
    return parser.parse_args()

def func(*args, **kwargs):
    a, b, c = args
    print 'a=', a
    print 'b=', b
    print 'c=', c
    for k, v in kwargs.iteritems():
        print '%s=' % k, v

if __name__ == '__main__':

    all_args = parse()
    ### need to split all_args into args and kwargs here ###
    func(*args, **kwargs)
Ezra Citron
  • 101
  • 2
  • 6
sammosummo
  • 495
  • 1
  • 7
  • 17
  • What exactly do you mean by "many times with different functions and different numbers of ... arguments"? I think you're going to have to provide an example of how you would call your function, along with what the script would do with the arguments passed. – chepner Jan 21 '16 at 19:11

1 Answers1

3

The Namespace you get back from parse_args will have attributes corresponding to each of your arguments. There will be no distinction between the positional arguments and the optionals, e.g.:

args
Namespace(a='1',b='one',x='foo', y=...)

Which, as is well documented, can be accessed as:

args.a
args.x
etc.

The Namespace can also be turned into a dictionary:

vars(args)
{'a'='1', 'b'='one', etc.}

You can pass the dictionary to a function as **kwargs. That's standard Python argument practice.

If you want to pass some arguments as *args you'll have to split those off the Namespace or dictionary yourself. Nothing in argparse will do that for you.

You could write a function like (not tested):

def split_args(args):
   vargs = vars(args)
   alist = ['a','b','c']
   args1 = []
   for a in alist:
        v = vargs.pop(a)
        args1.append(v)
   return args1, vars

Or more compactly, put the pop in a list comprehension:

In [702]: vargs = dict(a=1,b=3,c=4,x=5,y=3,z=3)
In [703]: [vargs.pop(a) for a in ['a','b','c']]
Out[703]: [1, 3, 4]
In [704]: vargs
Out[704]: {'y': 3, 'x': 5, 'z': 3}

In [705]: def foo(*args,**kwargs):
   .....:     print(args)
   .....:     print(kwargs)
   .....:     
In [706]: vargs = dict(a=1,b=3,c=4,x=5,y=3,z=3)
In [707]: foo(*[vargs.pop(a) for a in ['a','b','c']],**vargs)
(1, 3, 4)
{'x': 5, 'z': 3, 'y': 3}

Further idea using custom Action class

The parser determines whether an argument is an optional vs. positional by its option_strings attribute. add_argument returns an Action subclass, which will have attributes like:

MyAction(option_strings=[], dest='baz', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)

This is a positional because option_strings is an empty list.

MyAction(option_strings=['-m', '--mew'], dest='mew', nargs=None,...)

is an optional because that list is not empty.

The parser matches input strings with the option_strings and nargs, and then passes values to the __call__ method of the matching Action. This method is defined like:

def __call__(self, parser, namespace, values, option_string=None):
    setattr(namespace, self.dest, values)

This is the default store action. The values are put in the Namespace as the dest attribute.

The option_string parameter is the string that triggered this call, something like '-m' or '--mew', or None for a positional. The defined action types don't make use of this, but a user-defined Action class could do something.

class MyAction(argparse._StoreAction):
   def __call__(self, parser, namespace, values, option_string=None):
       # store option_string along with values in the Namespace
       setattr(namespace, self.dest, [values,option_string])

Or you could do something special with positionals, e.g.

       if option_string is None:
           # append values to a `star_args` attribute
           # rather than self.dest

With an action like this positionals could be accessed after parsing as:

args.star_args

The parser does maintain a list attribute like this. The extras that parse_known_args returns are stored temporarily in the Namespace in the '_UNRECOGNIZED_ARGS_ATTR' attribute.

Ezra Citron
  • 101
  • 2
  • 6
hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • 1
    I wondered whether there was some nice pythonic way to do the splitting. Is there nothing in argparse that differentiates optional and positional arguments? – sammosummo Jan 21 '16 at 19:24
  • There must be, because `print_help` knows. Look in `parser._action_groups` (dragons) – wim Jan 21 '16 at 19:29
  • 1
    @user1637894: Any particular reason you need to distinguish required and optional arguments when passing them to the internal function? There's nothing stopping you from providing required arguments as keyword arguments. – user2357112 Jan 21 '16 at 19:31
  • Actually, you can't always do that (an example is with `self` for methods). But I doubt that applies here. – wim Jan 21 '16 at 19:34
  • No reason other than curiosity. I've accepted your answer, but I would still like to know. – sammosummo Jan 21 '16 at 19:39
  • Internally `argparse` distinguishes between positionals and optionals. I could describe that reasoning if it helps. But that distinction is not used in the `Actions` that insert values into the `namespace`. You are trying to make `argparse` more like `optparse`. – hpaulj Jan 21 '16 at 19:41
  • To make it more like `optparse`, omit the positionals, and use `parse_known_args`. – hpaulj Jan 21 '16 at 19:44
  • Thanks, that's sufficient. Curiosity sated. – sammosummo Jan 21 '16 at 19:49
  • I have sketched an idea of how you could use a custom `Action` class to store all the `positionals` in a custom namespace attribute list. – hpaulj Jan 22 '16 at 08:25