classification
Title: argparse: successive parsing wipes out nargs=? values
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ajstewart, bethard, paul.j3, wolma
Priority: normal Keywords:

Created on 2016-11-18 15:41 by ajstewart, last changed 2017-07-29 00:02 by terry.reedy.

Messages (4)
msg281128 - (view) Author: Adam Stewart (ajstewart) * Date: 2016-11-18 15:41
I'm writing a wrapper that optionally accepts a file and reads more options from that file. The wrapper then needs to pass all of these options and the file to another program (qsub). Here is a minimal example to reproduce the behavior I'm seeing:

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-a')
>>> parser.add_argument('file', nargs='?')
>>> args = parser.parse_args(['-a', '3', 'myFile'])
>>> print(args)
Namespace(file='myFile', a='3')
>>> parser.parse_args(['-a', '4'], namespace=args)
>>> print(args)
Namespace(file=None, a='4')

The behavior I expect is that the file should remain as 'myFile', but it is being wiped out. Is there any way to prevent this, or is this actually a bug?

I can recreate this problem in Python 2.7 and 3.5.
msg281137 - (view) Author: Wolfgang Maier (wolma) * Date: 2016-11-18 16:16
try this:
parser.add_argument('file', nargs='?', default = argparse.SUPPRESS)

I don't think this is a bug. The default for the default parameter is None so the behavior is consistent with the documentation.
msg281138 - (view) Author: Adam Stewart (ajstewart) * Date: 2016-11-18 16:19
Works for me, thanks Wolfgang!
msg299354 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-07-27 23:13
The problem described here is restricted to `?` and `*' positionals, and is caused by the subtle way in which 'empty' optional positionals are handled.

The regular handling of defaults at the start of `parse_known_args` works fine.  The default is only written to the namespace if something isn't there already.

     if not hasattr(namespace, action.dest):
         if action.default is not SUPPRESS:
              setattr(namespace, action.dest, action.default)

But a positional with ? or * is always 'seen' because an empty list of strings satisfies the nargs pattern.  'get_values()' has special handling for this case:

        if not arg_strings and action.nargs == OPTIONAL:
            ....
                value = action.default

That is, it replaces the empty list with the 'default'. 

But take_action(), which does the actual saving, is conditional:

            # take the action if we didn't receive a SUPPRESS value
            # (e.g. from a default)
            if argument_values is not SUPPRESS:
                action(self, namespace, argument_values, option_string)

That explains why 'default=SUPPRESS' solves this issue.

It's enough to satisfy the OP's situation, but I don't think it's a robust fix.

I don't have a patch idea yet, but this probably should be reopened so there's a record of the potential problem.

More on the complications raised by these 'seen default actions' in 

http://bugs.python.org/issue18943
History
Date User Action Args
2017-07-29 00:02:53terry.reedysetnosy: + bethard
2017-07-27 23:13:28paul.j3setstatus: closed -> open
nosy: + paul.j3
messages: + msg299354

2016-11-18 16:19:02ajstewartsetstatus: open -> closed
resolution: not a bug
messages: + msg281138
2016-11-18 16:16:36wolmasetnosy: + wolma
messages: + msg281137
2016-11-18 15:41:15ajstewartcreate