classification
Title: Add suggestions to argparse error message output for unrecognized arguments
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: conchylicultor, paul.j3, rhettinger, shihai1991, xtreak
Priority: normal Keywords:

Created on 2019-12-20 08:56 by xtreak, last changed 2020-12-11 19:59 by paul.j3. This issue is now closed.

Messages (6)
msg358699 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-20 08:56
I came across this idea while working on error messages for click at https://github.com/pallets/click/issues/1446. Currently for unknown arguments which could in some case be typos argparse throws an error but doesn't make any suggestions. It could do some heuristic to suggest matches. The unrecognized argument error prints all unrecognized arguments so in that case it will be less useful to mix match suggestions. It can be helpful for single argument usages. argparse is performance sensitive since it's used in cli environments so I feel the tradeoff to do simple match to make suggestions as a good user experience.

# ssl_helper.py

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--include-ssl', action='store_true')
namespace = parser.parse_args()

No suggestions are included currently

$ python3.8 ssl_helper.py --include-ssll
usage: ssl_helper.py [-h] [--include-ssl]
ssl_helper.py: error: unrecognized arguments: --include-ssll

Include suggestions based when one of the option starts with the argument supplied similar to click

$ ./python.exe ssl_helper.py --include-ssll
usage: ssl_helper.py [-h] [--include-ssl]
ssl_helper.py: error: unrecognized argument: --include-ssll . Did you mean --include-ssl?

difflib.get_close_matches could also provide better suggestions in some cases but comes at import cost and could be imported only during error messages as proposed in the click issue

./python.exe ssl_helper.py --exclude-ssl
usage: ssl_helper.py [-h] [--include-ssl]
ssl_helper.py: error: unrecognized argument: --exclude-ssl . Did you mean --include-ssl?


Attached is a simple patch of the implementation with startswith which is more simple and difflib.get_close_matches

diff --git Lib/argparse.py Lib/argparse.py
index 5d3ce2ad70..e10a4f0c9b 100644
--- Lib/argparse.py
+++ Lib/argparse.py
@@ -1818,8 +1818,29 @@ class ArgumentParser(_AttributeHolder, _ActionsContainer):
     def parse_args(self, args=None, namespace=None):
         args, argv = self.parse_known_args(args, namespace)
         if argv:
-            msg = _('unrecognized arguments: %s')
-            self.error(msg % ' '.join(argv))
+            suggestion = None
+            if len(argv) == 1:
+                argument = argv[0]
+
+                # simple startswith
+                for option in self._option_string_actions:
+                    if argument.startswith(option):
+                        suggestion = option
+                        break
+
+                # difflib impl
+                import difflib
+                try:
+                    suggestion = difflib.get_close_matches(argv[0], self._option_string_actions, n=1)[0]
+                except IndexError:
+                    pass
+
+            if suggestion:
+                msg = _('unrecognized argument: %s . Did you mean %s?')
+                self.error(msg % (' '.join(argv), suggestion))
+            else:
+                msg = _('unrecognized arguments: %s')
+                self.error(msg % ' '.join(argv))
         return args
 
     def parse_known_args(self, args=None, namespace=None):
msg358758 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-12-21 06:25
-1  Given an unknown argument, we really can't know what the user intended.  The usage string already lists all available options and -h --help gives more detail when requested.
msg358866 - (view) Author: hai shi (shihai1991) * (Python triager) Date: 2019-12-25 14:08
I checked some other common clis and it is show all right available options too. So I thought the argparse's help function is good enough too ;)

```
$ ps -etest
error: TTY could not be found

Usage:
 ps [options]

 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.

For more details see ps(1).
```

```
$ top test
top: unknown option 't'
Usage:
  top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o field -w [cols]
```
msg358867 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-25 15:05
Thanks for the feedback. Closing it as rejected.
msg382870 - (view) Author: conchylicultor (conchylicultor) * Date: 2020-12-11 15:15
I don't think this should have been closed.

[1]  If the user is using sub_parser, the options are not even displayed. For example in our https://github.com/tensorflow/datasets project:

```
$ tfds build mnist --overwritte
usage: tfds [-h] [--helpfull] [--version] {build,new} ...
tfds: error: unrecognized arguments: --overwritte
```

[2] For some programs, there can be 20+ options and having to scroll through the list is not user friendly at all.

[3] Other CLI, like Google absl.flags has this option too and it is very convenient.
msg382881 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2020-12-11 19:59
In the subparser example, it's the `build` subparser that puts '--overwritte' in the unrecognized list.  That is then passed back to the main parser, which then issues the 'unrecognized' error, along with its own usage.

The subparser is called with `parse_known_args` while the proposed patch is run in the `parse_args` method of the main parser.  It doesn't have access to the subparser's arguments.  So implementing the proposed matching will be much harder.

For some types of error, such as type or choices, the subparser itself raises the error, with the appropriate usage.  

===

https://bugs.python.org/issue42297
[argparse] Bad error message formatting when using custom usage text

is another case where error messages produced by the subparser differ from messages produced by the main. In this case the unrecognized error usage message is clearer since it is produced by the main parser.

===

I didn't close this issue, but it does feel like an enhancement that's too big for the bug/issues forum.  The proposed patch could be developed as a separate 'parser.parse_args_with_hints' method, and distributed as a pypi addition.  During development and testing, the regular 'parser.parse_args()' does not need to be touched.
History
Date User Action Args
2020-12-11 19:59:07paul.j3setmessages: + msg382881
2020-12-11 15:15:11conchylicultorsetnosy: + conchylicultor
messages: + msg382870
2019-12-25 15:05:35xtreaksetstatus: open -> closed
resolution: rejected
messages: + msg358867

stage: resolved
2019-12-25 14:08:21shihai1991setnosy: + shihai1991
messages: + msg358866
2019-12-21 06:25:47rhettingersetmessages: + msg358758
2019-12-20 08:56:09xtreakcreate