This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author paul.j3
Recipients bethard, brett.cannon, paul.j3, quabla
Date 2016-03-11.04:00:41
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1457668844.47.0.267979899923.issue26394@psf.upfronthosting.co.za>
In-reply-to
Content
Let's see if I understand the proposed patch

Each call to 'take_action' is replaced with a save to a 'action_with_values' dictionary

For a flagged (optional) it saves the 'args' and 'option_string'; for positionals it save 'args':

-                take_action(action, args, option_string)
+                action_with_values(action)['argument_strings'] = args
+                action_with_values(action)['option_string'] = option_string

Then at the end of the 'consume_optional' and 'consume_positionals' loops, it processes all the deferred 'take_actions'

+        for action, kwargs in actions_with_values.items():
+            take_action(action, **kwargs)


It had previously looped through 'parser._actions' and created an entries in 'actions_with_values' with actions and 'fallback_strings' key.

In 'take_action', if the action's 'argument_strings' key (in actions_with_values) is None, it uses the 'fallback_strings' instead.

------------------

Has this patch been tested?  That is, does it pass all the tests in 'test_argparse.py'?

One potential failing is that the order in which 'take_action' occurs can change.  In the original, the order of 'take_action' depends on the order of occurrence in the sys.argv list.  With this change, the order depends on the hashing order in 'actions_with_values'.

In many cases that order does not matter.  But there's nothing, in the current code, to prevent order dependence.  Even if the user does not define custom Actions, actions like 'append' are execution order dependent.  And optionals may occur several times, e.g 

    python prog.py -f one --foo two -f three

the namespace could have (foo='three') or (foo=['one','two','three']) depending the Action class.

And different arguments can save to the same 'dest'.

Deferring 'take_action' like this is too fraught with backward compatibility issues to consider seriously.  Parsing is complex enough as it is, without adding the uncertainty of this differed 'take_action'.

Other qualms:

- On the surface the process of collecting 'fallback_strings' appears to handle the number of arguments correctly (with '_match_argument); but I have feeling it could be buggy.

- What if the 'fallback' wants to provide values and objects instead of strings?  That's a tricky enough issue when working with the defaults.  The example.py had to handle 'false' in a special way.  The 'store_const' actions (including 'store_true' and 'store_false') will need special testing.

- how will these changes behave with subparsers?  That is a big unknown.


--------------

An alternative place to apply this kind of 'fallback' is at the end of '_parse_known_args'.  At this point the parser has access to 'see_actions' and 'seen_non_default_actions' lists (or sets).  It uses those to test for 'required_actions' and required mutually_exclusive_groups.

In http://bugs.python.org/issue11588 (Add "necessarily inclusive" groups to argparse) I propose adding a 'hook' at this point that can be used apply more general group tests (not just the current xor, but all the other logical possibilities).  This hook could look at which actions were seen (i.e. acted on by take_action), and raise errors if the wrong combination was seen or not seen.  The idea is that access to 'see_actions' is more definitive than checking the namespace of 'is None' values.

I can imagine a fallback operating as part of this hook, filling in value for required actions that were not seen.

This is similar to looking at and modifying the args namespace after parsing, except that it has access to the 'seen_actions' set.  By acting at this point, the fallback is not constrained by the action's __call__ or any of its parameters (nargs, type, etc).  There are pros and cons to that.

-----------------------

I've mentioned else where (including SO questions) that 'ipython' uses argparse along with config.  It does so by first reading config, and then populating the parser with arguments derived from the config.  That way the user has several ways of setting values - the default config, profile configs, and the commandline.  In practice most values come from config, and only a select few get changed via commandline.

argparse already as a poor-man's config file mechanism, the 'fromfile_prefix_chars'.  This reads strings from a file and splices them into the 'argv' list.

       if self.fromfile_prefix_chars is not None:
            arg_strings = self._read_args_from_files(arg_strings)

It's not as powerful as config or reading the environment, but it still provides a way of adding strings to those already given in the commandline.

In short, I think it best to exhaustively look at alternatives that don't require surgery to the heart of parse_args.  A feature like this can only be added if it has absolutely zero chance of modifying the behavior of anyone else's parser.
History
Date User Action Args
2016-03-11 04:00:44paul.j3setrecipients: + paul.j3, brett.cannon, bethard, quabla
2016-03-11 04:00:44paul.j3setmessageid: <1457668844.47.0.267979899923.issue26394@psf.upfronthosting.co.za>
2016-03-11 04:00:44paul.j3linkissue26394 messages
2016-03-11 04:00:41paul.j3create