Python argparse - Add argument to multiple subparsers

Question

My script defines one main parser and multiple subparsers. I want to apply the -p argument to some subparsers. So far the code looks like this:

parser = argparse.ArgumentParser(prog="myProg")
subparsers = parser.add_subparsers(title="actions")

parser.add_argument("-v", "--verbose",
                    action="store_true",
                    dest="VERBOSE",
                    help="run in verbose mode")

parser_create = subparsers.add_parser ("create", 
                                        help = "create the orbix environment")
parser_create.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

# Update
parser_update = subparsers.add_parser ("update", 
                                        help = "update the orbix environment")
parser_update.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

As you can see the add_arument ("-p") is repeated twice. I actually have a lot more subparsers. Is there a way to loop through the existing subparsers in order to avoid repetition?

For the record, I am using Python 2.7

Related: http://stackoverflow.com/questions/14918804/how-to-show-help-for-all-subparsers-in-argparse — Francesco Frassinelli, May 18 '16 at 08:38
Also see example in documentation: [parents](https://docs.python.org/3/library/argparse.html#parents) — djvg, Jun 16 '22 at 12:36

score 102 · Accepted Answer · edited Sep 12 '20 at 15:57

102

This can be achieved by defining a parent parser containing the common option(s):

import argparse

parent_parser = argparse.ArgumentParser(description="The parent parser")
parent_parser.add_argument("-p", type=int, required=True,
                           help="set db parameter")
subparsers = parent_parser.add_subparsers(title="actions")
parser_create = subparsers.add_parser("create", parents=[parent_parser],
                                      add_help=False,
                                      description="The create parser",
                                      help="create the orbix environment")
parser_create.add_argument("--name", help="name of the environment")
parser_update = subparsers.add_parser("update", parents=[parent_parser],
                                      add_help=False,
                                      description="The update parser",
                                      help="update the orbix environment")

This produces help messages of the format:

parent_parser.print_help()

Output:

usage: main.py [-h] -p P {create,update} ...
The parent parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

parser_create.print_help()

Output:

usage: main.py create [-h] -p P [--name NAME] {create,update} ...
The create parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
  --name NAME      name of the environment
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

However, if you run your program, you will not encounter an error if you do not specify an action (i.e. create or update). If you desire this behavior, modify your code as follows.

<...>
subparsers = parent_parser.add_subparsers(title="actions")
subparsers.required = True
subparsers.dest = 'command'
<...>

This fix was brought up in this SO question which refers to an issue tracking a pull request.

update by @hpaulj

Due to changes in handling subparsers since 2011, it is a bad idea to use the main parser as a parent. More generally, don't try to define the same argument (same dest) in both main and sub parsers. The subparser values will overwrite anything set by the main (even the subparser default does this). Create separate parser(s) to use as parents. And as shown in the documentation, parents should use add_help=False.

edited Sep 12 '20 at 15:57

hpaulj

221,503
14
230
353

answered Sep 21 '11 at 11:19

Sven Marnach

574,206
118
941
841

17

From [the docs](https://docs.python.org/3/library/argparse.html#sub-commands): "when a help message is requested from a subparser, only the help for that particular parser will be printed. The help message will not include parent parser or sibling parser messages." That seems like a major shortcoming of this strategy. – Ryne Everett Apr 12 '14 at 13:09
I ended up adding a dummy argument to the root parser with a help message. – Ryne Everett Apr 12 '14 at 19:49
2

@RyneEverett: that manual part is confusing and might be outdated, because it seems at least in Python 3.5.3 subparser `--help` includes arguments from the parent parser. – user1338062 Sep 17 '18 at 05:57
I've edited the answer showing `--help` outputs for parent and child parsers. Notice the help message for `create` shows both "create" and all actions, which is incorrect. The looping answers do not suffer from this issue – craymichael Jun 04 '19 at 18:14
The idea seems correct, however the example code is somehow weird, because in this case one would not need any parent parsers and could add parameters directly to main parser – The Godfather Jun 14 '19 at 09:31
@TheGodfather Yes this is true, however you lose the hierarchy in doing so. There will be cases where you definitely do not want to mislead your user that e.g. the arguments for `create` can not be provided with the `update` command. – craymichael Jun 29 '19 at 07:01
@craymichael no-no, I mean that usually you still have other arguments and other subparsers and other root-level commands (see in my answer-example). Of course you still use parent, but only for the stuff it is actually a parent, not for every argument. In the given example `-p` should be argument only for two given subparsers, but not for the whole script (think of having 5 other subparsers which do not have this arg). – The Godfather Jul 01 '19 at 08:46
@TheGodfather I see what you are saying now. See my reply to your answer (shortly...). I will update this answer to reflect both options – craymichael Jul 02 '19 at 19:40
6

I think the reason it behaves oddly is that a "parent parser" is not actually the same thing as a "parser with subparsers". (note that the docs never show the two features used together) Pretty sure you need a top-level parser, a separate "parent" parser with the global args on it, and subparsers on the former with their `parents=` set to the latter. – Andrew Jul 16 '19 at 00:48
Could you please check my question https://stackoverflow.com/questions/59391084/python-multi-command-cli-with-common-options ? Probably you know the answer. Thanks. – Elephant Dec 18 '19 at 12:18
I'm running into an issue where I cannot get access to the help messages for a subcommand's options. I have a main parser, and a separate parent parser with `add_help=False` which holds arguments which are common to both subcommands which is added via `add_parser(..., parents=[parent_parser]`. When I then attempt to use `command subcommand -h` to view the help for those common args I get an `unrecognized arguments: -h` error. – ddejohn Jul 26 '22 at 17:54
@ddejohn If your setup is `global_p` (with all global args), `toplevel_p` (with parent `global_p`) and then `subparsers` (created from `toplevel_p`), and then parsers (added to `subparsers` with parent `global_p`), then the parser that requires `add_help=False` is actually JUST `global_p`, not `toplevel_p`. NOTE: if you do not own `global_p` or did not create/init `global_p` with `add_help=False` you can manually remove it by searching for the help action in `global_p._actions` and calling `global_p._remove_action()` on it. – davzaman Feb 01 '23 at 21:37

score 69 · Answer 2 · edited Aug 28 '19 at 12:55

69

The accepted answer is correct; the proper way is to use parent parsers. However, the example code IMO was not really solving the problem. Let me add my few cents to provide a more suitable example.

The main difference with accepted answer is the explicit possibility to have some root-level arguments (like --verbose) as well as shared arguments only for some subparsers (-p only for the create and update subparsers but not for others)

# Same main parser as usual
parser = argparse.ArgumentParser()

# Usual arguments which are applicable for the whole script / top-level args
parser.add_argument('--verbose', help='Common top-level parameter',
                    action='store_true', required=False)

# Same subparsers as usual
subparsers = parser.add_subparsers(help='Desired action to perform', dest='action')

# Usual subparsers not using common options
parser_other = subparsers.add_parser("extra-action", help='Do something without db')

# Create parent subparser. Note `add_help=False` and creation via `argparse.`
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument('-p', help='add db parameter', required=True)

# Subparsers based on parent

parser_create = subparsers.add_parser("create", parents=[parent_parser],
                                      help='Create something')
# Add some arguments exclusively for parser_create

parser_update = subparsers.add_parser("update", parents=[parent_parser],
                                      help='Update something')
# Add some arguments exclusively for parser_update

This is the top-level help message (note that -p parameter is not shown here, which is exactly what you would expect—because it is specific to some subparsers):

>>> parser.print_help()
usage: [-h] [--verbose] {extra-action,create,update} ...

positional arguments:
  {extra-action,create,update}
                        Desired action to perform
    extra-action        Do something without db
    create              Create something
    update              Update something

optional arguments:
  -h, --help            show this help message and exit
  --verbose             Common top-level parameter

And the help message for the create action:

>>> parser_create.print_help()
usage:  create [-h] -p P

optional arguments:
  -h, --help  show this help message and exit
  -p P        add db parameter

edited Aug 28 '19 at 12:55

Michael

8,362
6
61
88

answered Jun 14 '19 at 09:51

The Godfather

4,235
4
39
61

So, in my previous edit to the main answer, I include the "-p" option in the `parent` parser to show how users may have common options between all subparsers. I then showed that a "--name" option could be added to a single `subparser` that is unique to the `create` command/action. The main difference in our solutions is in how help is shown. Here is [my output](https://pastebin.com/0bEMrMzV) and here is [your output](https://pastebin.com/kxN25Zv1). – craymichael Jul 02 '19 at 19:53
@craymichael you do not need parent parser to add arguments for *all* (this can be done just with normal parser) ;) Different help is just an illustration for much bigger difference here. BTW I also added help output to my answer. – The Godfather Jul 03 '19 at 20:19
1

I completely agree, it is not needed but some may want to use a mix of both. Your help messages are definitely cleaner, thanks for adding! – craymichael Jul 04 '19 at 17:51
1

The separate parent parser is the only way to share positional arguments between multiple subcommands. – Paul Fenney Aug 14 '19 at 09:17
4

This is a brilliantly clear example, this answer needs more upvotes folks. – David Parks Nov 25 '19 at 16:53
Should this work in this way: `--verbose create -p 2 update -p 3`. For me it says: "usage: argparser.py [-h] [--verbose] {extra-action,create,update} ... argparser.py: error: unrecognized arguments: update" – Elephant Dec 17 '19 at 16:07
1

@Elephant no, subparser's actions are mutually exclusive. First you run `--verbose create -p 2` and then you run `--verbose update -p 3` – The Godfather Mar 01 '20 at 22:55
how come parent_parses is included in arguments if parent_parser.parse_args() was never done? – cikatomo Apr 23 '21 at 07:43
@cikatomo I don't really get what you mean, `print_help` just prints help and doesn't require calling `parse_args`. But in the product you'd use `parser.parse_args()` (or `parent_parser`) – The Godfather Apr 23 '21 at 16:31
@TheGodfather yes, that's right, what I mean is if we do parser.parse_args(), then how come we can also use '-p' argument from parent_parser without ever doing parent_parser.parse_args()? – cikatomo Apr 23 '21 at 16:42
I am trying to avoid using another library but let me get if I understand this. We have to create separate parser to use it as parent in subparser of the original parser. That's the only way to get the proper print help. And we still can't use `--verbose` at the end. It has to be at the beginning? – cikatomo Apr 23 '21 at 17:38

score 17 · Answer 3 · answered Aug 26 '15 at 09:40

You can also loop over the subparsers and add the same option to all of them.

parser = argparse.ArgumentParser(prog="myProg")
subparsers = parser.add_subparsers(title="actions")
parser.add_argument("-v", "--verbose",
                    action="store_true",
                    dest="VERBOSE",
                    help="run in verbose mode")

parser_create = subparsers.add_parser ("create", 
                                        help = "create the orbix environment")
parser_update = subparsers.add_parser ("update", 
                                        help = "update the orbix environment")

for subparser in [parser_create, parser_update]:
    subparser.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

i think that's a bad aproach, cause you need to loop over parameters. it's more DRY and redable the Sven Marnach answer. — NeoMorfeo, Oct 21 '15 at 11:00

score 10 · Answer 4 · edited Jun 13 '19 at 02:13

10

You can loop over your subparsers in the following way.

for name, subp in subparsers.choices.items():
    print(subp)
    subp.add_argument(dest='g', help='Input for g variable', default=7, type=int)

Note that by using subparsers.choices that you avoid needing to hard-code all of the subparsers.

edited Jun 13 '19 at 02:13

craymichael

4,578
1
15
24

answered Oct 12 '17 at 18:31

Gerard Kool

129
1
4

This is the same as @JanK's answer – craymichael Jun 03 '19 at 22:24
4

@craymichael Not quite. It is similar, i'll give you that. The difference lies in the iterable. JanK's answer uses a list of parsers, mine uses subparsers.choices.items(). The advantage of subparsers.choices.items()is that subparsers.choices.items() always contains all used subparsers. When using a list you will have to change it when subparsers are added or removed. I do have to give Jank's credit for the idea, i just took it a step further because i dont like to hardcode and maintain the list. – Gerard Kool Jun 12 '19 at 19:44
@Gerald Kool Ah, I see. I missed that when looking this answer over! Thank you – craymichael Jun 12 '19 at 22:28
This works well for 1 level parsers, but recursively grabbing subparsers can be tough if you have a deeply nested command tree. I created some helper functions to make this easier here: https://github.com/turtlemonvh/argparse-subparser-filter – turtlemonvh May 06 '20 at 14:29
1

I like this one because I can write `--verbose` at the end, not beginning – cikatomo Apr 23 '21 at 17:48
@turtlemonvh you say to write `p = generate_parsers()`, but where is that function in your code? – cikatomo Apr 23 '21 at 17:49
`generate_parser` isn't a function in my code. It's just an example of a function you would write that returns a `argparse.ArgumentParser` object. You don't need a function at all - you just need a `argparse.ArgumentParser` that you can use as an argument to the `get_parsers` function in my code. – turtlemonvh Apr 24 '21 at 18:26

Python argparse - Add argument to multiple subparsers

4 Answers4

update by @hpaulj

Linked

Related