classification
Title: Improve exception message in ast.literal_eval
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: 32892 32893 Superseder:
Assigned To: Nosy List: Rosuav, brian.curtin, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2018-02-20 17:26 by Rosuav, last changed 2018-07-03 17:56 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 340 open Rosuav, 2018-02-20 17:37
Messages (6)
msg312423 - (view) Author: Chris Angelico (Rosuav) * Date: 2018-02-20 17:26
When a non-literal is given to literal_eval, attempt to be more
helpful with the message, rather than calling it 'malformed'.
msg312424 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-02-20 18:00
I afraid this makes an error message more confusing and misleading.

>>> ast.literal_eval('~2')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
    return _convert(node_or_string)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 92, in _convert
    return _convert_signed_num(node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
    return _convert_num(node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
    raise ValueError('%s not allowed in literal' % type(node).__name__)
ValueError: UnaryOp not allowed in literal

This is not true since + and - are allowed:

>>> ast.literal_eval('-2')
-2

>>> ast.literal_eval('2+3')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
    return _convert(node_or_string)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 92, in _convert
    return _convert_signed_num(node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
    return _convert_num(node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
    raise ValueError('%s not allowed in literal' % type(node).__name__)
ValueError: BinOp not allowed in literal

But:

>>> ast.literal_eval('2+3j')
(2+3j)

>>> ast.literal_eval('"a"+"b"')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
    return _convert(node_or_string)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 85, in _convert
    left = _convert_signed_num(node.left)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
    return _convert_num(node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
    raise ValueError('%s not allowed in literal' % type(node).__name__)
ValueError: Str not allowed in literal

But Str is allowed:

>>> ast.literal_eval('"ab"')
'ab'
msg312427 - (view) Author: Chris Angelico (Rosuav) * Date: 2018-02-20 18:05
Hmm, I think I see what I broke there. It was part of the merge conflict resolution - I moved the check into the function, which is actually incorrect. It wasn't misleading like that in the original patch. Will fix that.
msg312429 - (view) Author: Chris Angelico (Rosuav) * Date: 2018-02-20 18:11
Actually, it's a bit more complicated than I thought. Current proposed solution: Track the context of each conversion, thus allowing different errors to be distinguished.
msg312442 - (view) Author: Chris Angelico (Rosuav) * Date: 2018-02-20 22:48
(BTW, by "proposed" I mean that the change that I describe is in the PR.)
msg320993 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-03 17:56
Please wait until issue32892 and issue32893 be committed or rejected before merging this PR.
History
Date User Action Args
2018-07-03 17:56:15serhiy.storchakasetnosy: + brian.curtin
dependencies: + Remove specific constant AST types in favor of ast.Constant, ast.literal_eval() shouldn't accept booleans as numbers in AST
messages: + msg320993
2018-02-24 13:54:05serhiy.storchakasetpull_requests: - pull_request5555
2018-02-20 22:48:48Rosuavsetmessages: + msg312442
2018-02-20 18:11:23Rosuavsetmessages: + msg312429
2018-02-20 18:05:12Rosuavsetmessages: + msg312427
2018-02-20 18:00:42serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg312424
2018-02-20 17:37:12Rosuavsetkeywords: + patch
stage: patch review
pull_requests: + pull_request5558
2018-02-20 17:26:21Rosuavcreate