Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ast.literal_eval does not properly handled complex numbers #49157

Closed
mitsuhiko opened this issue Jan 10, 2009 · 17 comments
Closed

ast.literal_eval does not properly handled complex numbers #49157

mitsuhiko opened this issue Jan 10, 2009 · 17 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@mitsuhiko
Copy link
Member

BPO 4907
Nosy @birkenfeld, @rhettinger, @terryjreedy, @mdickinson, @benjaminp, @mitsuhiko
Files
  • literal-eval.patch: broken patch
  • literal-eval.patch
  • literal-eval.patch: final patch with unittests
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2009-01-13.11:52:55.417>
    created_at = <Date 2009-01-10.15:45:02.291>
    labels = ['type-bug', 'library']
    title = 'ast.literal_eval does not properly handled complex numbers'
    updated_at = <Date 2010-10-08.00:52:56.767>
    user = 'https://github.com/mitsuhiko'

    bugs.python.org fields:

    activity = <Date 2010-10-08.00:52:56.767>
    actor = 'rhettinger'
    assignee = 'none'
    closed = True
    closed_date = <Date 2009-01-13.11:52:55.417>
    closer = 'aronacher'
    components = ['Library (Lib)']
    creation = <Date 2009-01-10.15:45:02.291>
    creator = 'aronacher'
    dependencies = []
    files = ['12674', '12675', '12707']
    hgrepos = []
    issue_num = 4907
    keywords = ['patch']
    message_count = 17.0
    messages = ['79548', '79550', '79554', '79604', '79701', '79705', '79723', '79730', '79733', '79735', '79736', '79737', '79738', '79739', '79740', '80004', '118154']
    nosy_count = 7.0
    nosy_names = ['georg.brandl', 'rhettinger', 'terry.reedy', 'mark.dickinson', 'benjamin.peterson', 'gpolo', 'aronacher']
    pr_nums = []
    priority = 'normal'
    resolution = 'accepted'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue4907'
    versions = ['Python 2.6']

    @mitsuhiko
    Copy link
    Member Author

    ast.literal_eval does not properly handle complex numbers:

    >>> ast.literal_eval("1j")
    1j
    >>> ast.literal_eval("2+1j")
    Traceback (most recent call last):
      ...
    ValueError: malformed string
    >>> ast.literal_eval("(2+1j)")
    Traceback (most recent call last):
      ...
    ValueError: malformed string

    Expected result:

    >>> ast.literal_eval("1j")
    1j
    >>> ast.literal_eval("2+1j")
    (2+1j)
    >>> ast.literal_eval("(2+1j)")
    (2+1j)

    I attached a patch that fixes this problem.

    @mitsuhiko mitsuhiko added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jan 10, 2009
    @mitsuhiko
    Copy link
    Member Author

    fixed patch :)

    @benjaminp
    Copy link
    Contributor

    Looks good to me assuming you add a test.

    @mdickinson
    Copy link
    Member

    I'm not sure that this is desirable behaviour. There's no such thing as a
    complex literal---only imaginary literals. Why allow evaluation of 2+1j
    but not of 2 + 1, or 2*1j.

    In any case, I'm not sure that the patch behaves as intended. For
    example,

    >>> ast.literal_eval('2 + (3 + 4j)')
    (5+4j)

    @mdickinson
    Copy link
    Member

    BTW, both those "I'm not sure"s should be taken literally: I'm not a user
    of the ast module, I don't know who the typical users are, and I don't
    know what the typical uses for the literal_eval function are. The patch
    just struck me as odd, so I thought I'd comment.

    IOW, take my comments with a large pinch of salt.

    @mitsuhiko
    Copy link
    Member Author

    Here a patch with unittests to correctly handle complex numbers. This
    does not allow the user of arbitrary add/sub expressions on complex numbers.

    @birkenfeld
    Copy link
    Member

    Nit: the "except" should only catch ValueError.

    @mdickinson
    Copy link
    Member

    Nice fix!

    Exactly which complex strings should be accepted here?
    The patched version of literal-eval still accepts some
    things that would be rejected as inputs to complex():

    >>> ast.literal_eval('-1+-3j')
    (-1-3j)
    >>> complex('-1+-3j')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: complex() arg is a malformed string

    and it produces different results from the complex
    constructor in some other cases:

    >>> complex('-0.0+0.0j').real
    -0.0
    >>> ast.literal_eval('-0.0+0.0j').real
    0.0

    But since I don't really know what ast_literal
    is used for, I don't know whether this matters.

    It still seems odd to me to be doing just one
    special case of expression evaluation in
    ast_literal, but maybe that's just me.

    @mitsuhiko
    Copy link
    Member Author

    literal_eval has eval() semantics and not complex() constructor
    semantics. It accepts what eval() accepts just without arithmetic and
    unsafe features.

    For exmaple "(2 + 4j)" is perfectly fine even though the complex call
    only supports "2+4j" (no parentheses and whitespace).

    I commit the fix with the ValueError except Georg suggested.

    @mdickinson
    Copy link
    Member

    So why accept (4+2j) but not (2j+4)?

    (BTW, I'm fairly sure that the complex constructor does
    accept parentheses; you're right about the whitespace, though.)

    @mitsuhiko
    Copy link
    Member Author

    Indeed, it accepts parentheses in 2.6 now, but not in 2.5 or earlier.

    Why not the other way round? Somewhere there has to be a limit. And if
    you write down complex numbers you usually have the imaginary part after
    the real part.

    But let's try no to make this a bikeshed discussion. If you say that
    literal_eval can safely evaluate the repr() of builtins (with the
    notable exception of reprs that eval can't evaluate either [like nan,
    inf etc.]) and probably a bit more it should be fine :)

    @mitsuhiko
    Copy link
    Member Author

    Fixed in rev68571.

    @gpolo
    Copy link
    Mannequin

    gpolo mannequin commented Jan 13, 2009

    Why didn't you use assertRaises in place of that try/except for a test ?

    I was somewhat following this issue and just saw it being commited, but
    the change was being discussed. Aren't you supposed to commit these kind
    of changes only after entering in agreement with others ?

    @mdickinson
    Copy link
    Member

    If you say that
    literal_eval can safely evaluate the repr() of builtins

    Sorry, yes, that makes perfect sense. (And now I see that that's what
    distinguishes 4+2j from 2j+4---finally the light dawns.) Apologies for
    being obtuse.

    @mitsuhiko
    Copy link
    Member Author

    Why didn't you use assertRaises in place of that try/except for a test ?
    Could be changed.

    I was somewhat following this issue and just saw it being commited,
    but the change was being discussed. Aren't you supposed to commit
    these kind of changes only after entering in agreement with others ?
    The "needs review" keyowrd was removed, I was under the impression I can
    commit now :)

    @terryjreedy
    Copy link
    Member

    I assume from the discussion that the patch was accepted/committed and
    changed the resolution and stage field to match.

    FWIW, list displays, for instance, are not literals either but are
    successfully evaluated, so doing same for complex 'displays' seems
    sensible to me too, and in line with the purpose of the method.

    @rhettinger
    Copy link
    Contributor

    Fixed handling on unary minus in r85314. In so doing, it also liberalized what literal_eval() accepts (3j+4 for example). This simplified the implementation and removed an unnecessary restriction which wasn't needed for "safety".

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants