Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse crashes in gettext when processing missing arguments #79966

Closed
efahl mannequin opened this issue Jan 19, 2019 · 7 comments
Closed

argparse crashes in gettext when processing missing arguments #79966

efahl mannequin opened this issue Jan 19, 2019 · 7 comments
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@efahl
Copy link
Mannequin

efahl mannequin commented Jan 19, 2019

BPO 35785
Nosy @efahl, @asottile, @corona10
Files
  • foo.mo: Binary message translation file, put in 'en_US/LC_MESSAGES/'
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-01-20.17:24:13.992>
    created_at = <Date 2019-01-19.17:38:06.276>
    labels = ['3.7', 'library', 'type-crash']
    title = 'argparse crashes in gettext when processing missing arguments'
    updated_at = <Date 2019-01-20.17:24:13.991>
    user = 'https://github.com/efahl'

    bugs.python.org fields:

    activity = <Date 2019-01-20.17:24:13.991>
    actor = 'eric.fahlgren'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-01-20.17:24:13.992>
    closer = 'eric.fahlgren'
    components = ['Library (Lib)']
    creation = <Date 2019-01-19.17:38:06.276>
    creator = 'eric.fahlgren'
    dependencies = []
    files = ['48068']
    hgrepos = []
    issue_num = 35785
    keywords = []
    message_count = 7.0
    messages = ['334065', '334069', '334074', '334076', '334077', '334080', '334093']
    nosy_count = 4.0
    nosy_names = ['bethard', 'eric.fahlgren', 'Anthony Sottile', 'corona10']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue35785'
    versions = ['Python 3.7']

    @efahl
    Copy link
    Mannequin Author

    efahl mannequin commented Jan 19, 2019

    When argparse is configured with an option that takes arguments, then the script is invoked with the switch but no arguments, a nonsensical exception is raised during gettext processing.

    In the 3.7.1 source, the error is at line 2077 of argparse.py, where 'action.nargs' is not an integer as expected by 'ngettext', but one of None, '*' or '?':

                default = ngettext('expected %s argument',
                                   'expected %s arguments',
                                   action.nargs) % action.nargs
                msg = nargs_errors.get(action.nargs, default)

    Fix should be pretty trivial, swap the two lines and if 'get' produces None, only then compute the default.

    File "C:\Program Files\Python37\lib\argparse.py", line 1749, in parse_args
    args, argv = self.parse_known_args(args, namespace)
    File "C:\Program Files\Python37\lib\argparse.py", line 1781, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
    File "C:\Program Files\Python37\lib\argparse.py", line 1987, in _parse_known_args
    start_index = consume_optional(start_index)
    File "C:\Program Files\Python37\lib\argparse.py", line 1917, in consume_optional
    arg_count = match_argument(action, selected_patterns)
    File "C:\Program Files\Python37\lib\argparse.py", line 2079, in _match_argument
    action.nargs) % action.nargs
    File "C:\Program Files\Python37\lib\gettext.py", line 631, in ngettext
    return dngettext(current_domain, msgid1, msgid2, n)
    File "C:\Program Files\Python37\lib\gettext.py", line 610, in dngettext
    return t.ngettext(msgid1, msgid2, n)
    File "C:\Program Files\Python37\lib\gettext.py", line 462, in ngettext
    tmsg = self._catalog[(msgid1, self.plural(n))]
    File "<string>", line 4, in func
    File "C:\Program Files\Python37\lib\gettext.py", line 168, in _as_int
    (n.__class
    _.__name__,)) from None
    TypeError: Plural value must be an integer, got NoneType

    @efahl efahl mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump labels Jan 19, 2019
    @asottile
    Copy link
    Mannequin

    asottile mannequin commented Jan 19, 2019

    Can you provide a reproducer? I'm having difficulty reproducing with this script:

    import argparse
    
    p = argparse.ArgumentParser()
    p.add_argument('--foo', nargs=None)
    args = p.parse_args()
    $ python3.7 --version
    Python 3.7.2
    $ python3.7 t.py --foo
    usage: t.py [-h] [--foo FOO]
    t.py: error: argument --foo: expected one argument

    @efahl
    Copy link
    Mannequin Author

    efahl mannequin commented Jan 19, 2019

    After a bit more digging, it's a side effect of having the locale set with 'Plural-Forms'. I've attached the resulting .mo file, but since it's a binary, I'm not sure it will work cross-platform, so here's how to recreate it.

    cat en_US/LC_MESSAGES/foo.po
    msgid ""
    msgstr "Plural-Forms: nplurals=2; plural=(n != 1);\n"

    python /Python37/Tools/i18n/msgfmt.py en_US/LC_MESSAGES/foo.po
    ll en_US/LC_MESSAGES/
    -rwx------+ 1 efahlgren Domain Users 89 2019-01-19 14:36 foo.mo*
    -rw-r--r--+ 1 efahlgren Domain Users 69 2019-01-19 14:34 foo.po

    Then you can reproduce with some setup in your script:

    import os
    import gettext
    import argparse
    
    os.putenv('LANG', 'en_US')  # Just to make sure.
    gettext.bindtextdomain('foo', '.')
    gettext.textdomain('foo')
    
    p = argparse.ArgumentParser()
    p.add_argument('--foo', nargs=None)
    p.parse_args()

    @corona10
    Copy link
    Member

    No crash at Python 3.7.2+ (heads/3.7:47290e7642, Jan 20 2019, 12:22:44)
    and master branch

    @corona10
    Copy link
    Member

    Umm looks like I should pass an argument on cmd to reproduce it. I will try to reproduce it later

    @corona10
    Copy link
    Member

    ➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo test
    ➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo 1
    ➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo
    usage: bpo-35785.py [-h] [--foo FOO]
    bpo-35785.py: error: argument --foo: expected one argument

    @efahl
    Copy link
    Mannequin Author

    efahl mannequin commented Jan 20, 2019

    Thanks, I installed 3.7.2 on one of our non-production machines and it appears that gettext has been fixed, so I'm closing this.

    python -V
    Python 3.7.2
    python bpo-35785.py --foo
    usage: bpo-35785.py [-h] [--foo FOO]
    bpo-35785.py: error: argument --foo: expected one argument

    @efahl efahl mannequin closed this as completed Jan 20, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant