Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyLong_FromString documentation wrong on numbers with leading zero and base=0 #73937

Closed
cubinator mannequin opened this issue Mar 7, 2017 · 13 comments
Closed

PyLong_FromString documentation wrong on numbers with leading zero and base=0 #73937

cubinator mannequin opened this issue Mar 7, 2017 · 13 comments
Labels
3.7 (EOL) end of life docs Documentation in the Doc dir type-feature A feature request or enhancement

Comments

@cubinator
Copy link
Mannequin

cubinator mannequin commented Mar 7, 2017

BPO 29751
Nosy @terryjreedy, @mdickinson, @vadmium, @Mariatta, @cubinator, @csabella
PRs
  • bpo-29751: DOC: PyLong_FromString wrong on number with leading zero a… #915
  • [3.6] bpo-29751: Improve PyLong_FromString documentation (GH-915) #1266
  • [3.5] bpo-29751: Improve PyLong_FromString documentation (GH-915) #1267
  • bpo-29751: add Cheryl Sabella to Misc/ACKS #1268
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-04-24.04:06:38.713>
    created_at = <Date 2017-03-07.21:27:48.248>
    labels = ['type-feature', '3.7', 'docs']
    title = 'PyLong_FromString documentation wrong on numbers with leading zero and base=0'
    updated_at = <Date 2017-04-24.09:33:13.892>
    user = 'https://github.com/cubinator'

    bugs.python.org fields:

    activity = <Date 2017-04-24.09:33:13.892>
    actor = 'cheryl.sabella'
    assignee = 'docs@python'
    closed = True
    closed_date = <Date 2017-04-24.04:06:38.713>
    closer = 'Mariatta'
    components = ['Documentation']
    creation = <Date 2017-03-07.21:27:48.248>
    creator = 'cubinator'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 29751
    keywords = []
    message_count = 13.0
    messages = ['289188', '289190', '289219', '290877', '290879', '290981', '291019', '292189', '292190', '292191', '292192', '292193', '292214']
    nosy_count = 7.0
    nosy_names = ['terry.reedy', 'mark.dickinson', 'docs@python', 'martin.panter', 'Mariatta', 'cubinator', 'cheryl.sabella']
    pr_nums = ['915', '1266', '1267', '1268']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue29751'
    versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

    @cubinator
    Copy link
    Mannequin Author

    cubinator mannequin commented Mar 7, 2017

    Calling PyLong_FromString(str, NULL, 0) fails, if str is a string containing a decimal number with leading zeros, even though such strings should be parsed as decimal numbers according to the documentation:

    "If base is 0, the radix will be determined based on the leading characters of str: if str starts with '0x' or '0X', radix 16 will be used; if str starts with '0o' or '0O', radix 8 will be used; if str starts with '0b' or '0B', radix 2 will be used; otherwise radix 10 will be used"

    Examples:
    PyLong_FromString("15", NULL, 0); // Returns int(15) (Correct)
    PyLong_FromString("0xF", NULL, 0); // Returns int(15) (Correct)
    PyLong_FromString("015", NULL, 0); // Should return int(15), but raises ValueError: invalid literal for int() with base 0: '015'

    Version information:
    Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32

    @cubinator cubinator mannequin added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Mar 7, 2017
    @vadmium
    Copy link
    Member

    vadmium commented Mar 7, 2017

    My guess is this is supposed to emulate (or is actually the implementation of) the "int" constructor and the Python syntax. In these cases, numbers with leading zeros are disallowed. This was to help with Python 2 porting, where a leading zero specified an octal number.

    >>> 010
        010
          ^
    SyntaxError: invalid token
    >>> int("010", 0)
    ValueError: invalid literal for int() with base 0: '010'

    Maybe it is better to fix the documentation.

    @serhiy-storchaka serhiy-storchaka added docs Documentation in the Doc dir and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Mar 7, 2017
    @serhiy-storchaka serhiy-storchaka added type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels Mar 7, 2017
    @mdickinson
    Copy link
    Member

    Yes, PyLong_FromString is directly used by the implementation of int, and is also used in parsing of numeric integer literals in source:

    return PyLong_FromString(s, (char **)0, 0);

    So I agree that this is a documentation bug. There's also no mention of the support for underscores in the documentation.

    @brettcannon brettcannon changed the title PyLong_FromString fails on decimals with leading zero and base=0 PyLong_FromString documentation wrong on numbers with leading zero and base=0 Mar 9, 2017
    @csabella
    Copy link
    Contributor

    I have a pull request ready for the documentation, but I didn't understand the underscore usage, so I couldn't add that.

    If you explain it, then I can try to add it.

    @Mariatta Mariatta added the 3.7 (EOL) end of life label Mar 30, 2017
    @terryjreedy
    Copy link
    Member

    String arguments to int are quoted int literals. From
    https://docs.python.org/3/reference/lexical_analysis.html#literals
    'Underscores are ignored for determining the numeric value of the literal. They can be used to group digits for enhanced readability. One underscore can occur between digits, and after base specifiers like 0x.'

    For your patch, I would summarize this by expanding 'Leading spaces are ignored.' to the following (in patch comment also).
    "Leading spaces and single underscores after a base specifier and between digits are ignored."

    @csabella
    Copy link
    Contributor

    csabella commented Apr 1, 2017

    Thank you. I've added that change.

    For the backporting, I think that would only be applicable to 3.6 and 3.7?

    @vadmium
    Copy link
    Member

    vadmium commented Apr 2, 2017

    Underscores are only applicable to 3.6+, but the original concern about leading zeros applies to 3.5.

    On Git Hub I suggested dropping the details and just referring to the Lexical Analysis section <https://docs.python.org/3.5/reference/lexical_analysis.html#integer-literals\> for the details.

    FWIW here is my understanding of integer literals (with base=0):

    • 0x10 => hexadecimal
    • 0b10 => binary
    • 0o10 => octal (corresponds to 010 in Python 2)
    • 01 => illegal (avoids conflict with Python 2)
    • 00 => zero (special case; was treated as octal zero in Python 2)
    • 10 => decimal (must not start with digit 0)

    If you want to spell out the rules, in my mind there are four special prefixes, 0x, 0b, 0o and 0, and the default is decimal if none of those prefixes apply.

    @Mariatta
    Copy link
    Member

    New changeset 26896f2 by Mariatta (csabella) in branch 'master':
    bpo-29751: Improve PyLong_FromString documentation (GH-915)
    26896f2

    @Mariatta
    Copy link
    Member

    New changeset d51d093 by Mariatta in branch '3.5':
    [3.5] bpo-29751: Improve PyLong_FromString documentation (GH-915) (bpo-1267)
    d51d093

    @Mariatta
    Copy link
    Member

    New changeset ea0efa3 by Mariatta in branch '3.6':
    [3.6] bpo-29751: Improve PyLong_FromString documentation (GH-915) (bpo-1266)
    ea0efa3

    @Mariatta
    Copy link
    Member

    New changeset 9eb5ca0 by Mariatta in branch 'master':
    bpo-29751: add Cheryl Sabella to Misc/ACKS (GH-1268)
    9eb5ca0

    @Mariatta
    Copy link
    Member

    I merged the PR, backported it to 3.5 and 3.6, and added Cheryl to Misc/ACKS.

    Thanks everyone :)

    @csabella
    Copy link
    Contributor

    Oh, I didn't expect that. That is so cool! Thanks Mariatta. :-)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life docs Documentation in the Doc dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants