classification
Title: literal_eval raises SystemError instead of SyntaxError, because of illegal escape sequence
Type: behavior Stage: resolved
Components: Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, brett.cannon, davidhalter, ezio.melotti, georg.brandl, ncoghlan, python-dev
Priority: normal Keywords:

Created on 2012-09-02 08:46 by davidhalter, last changed 2012-09-02 18:25 by python-dev. This issue is now closed.

Messages (3)
msg169687 - (view) Author: David Halter (davidhalter) Date: 2012-09-02 08:46
Hi! I'm the developer of Jedi: https://github.com/davidhalter/jedi, which is an auto-completion library. I try to support Python 2.5-3.2.
The bug just appears in Python 3.2.

Parsing string literals with literal_eval works normally without a problem, but in this certain constellation, it raises a SystemError (instead of a SyntaxError).

The error line is simple:
literal_eval(r"'\U'")

But: It only raises the error in that particular place:
https://github.com/davidhalter/jedi/blob/master/parsing.py#L157

To try and test it:

git clone git://github.com/davidhalter/jedi.git
cd jedi
# uncomment the line after TODO in parsing.Scope.add_docstr
python3 test/run.py array

The error message:

Traceback (most recent call last):
  File "./run.py", line 51, in run_definition_test
    result = defs(line_nr, len(line))
  File "./run.py", line 49, in defs
    return set(functions.get_definitions(source, line_nr, indent, path))
  File "./functions.py", line 253, in get_definitions
    scopes = _prepare_goto(source, pos, source_path, f, goto_path)
  File "./functions.py", line 227, in _prepare_goto
    scopes = evaluate.follow_statement(stmt)
  File "./helpers.py", line 23, in __call__
    if self.push_stmt(stmt):
  File "./helpers.py", line 31, in push_stmt
    self.current = RecursionNode(stmt, self.current)
  File "./helpers.py", line 75, in __init__
    or (self.script == builtin.Builtin.scope)
  File "./builtin.py", line 408, in scope
    return self._builtins.parser.module
  File "./builtin.py", line 42, in parser
    self._load_module()
  File "./builtin.py", line 50, in _load_module
    self._parser = parsing.PyFuzzyParser(source, self.path or self.name)
  File "./parsing.py", line 1047, in __init__
    self.parse()
  File "./parsing.py", line 1600, in parse
    stmt, tok = self._parse_statement(self.current)
  File "./parsing.py", line 1386, in _parse_statement
    self.scope.add_docstr(self.last_token[1])
  File "./parsing.py", line 157, in add_docstr
    literal_eval(r"'\U'")
  File "/usr/lib/python3.2/ast.py", line 48, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python3.2/ast.py", line 36, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
SystemError: ../Objects/tupleobject.c:126: bad argument to internal function

Cheers!
David
msg169688 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-09-02 09:22
You can try to remove things until that error goes away to see what is causing the problem and/or print the values of source, filename, mode, and PyCF_ONLY_AST to see if there's anything unusual.
msg169717 - (view) Author: Roundup Robot (python-dev) Date: 2012-09-02 18:25
New changeset 0db75a55145a by Benjamin Peterson in branch '3.2':
prevert ast errors from being normalized before ast_error_finish is called (closes #15846)
http://hg.python.org/cpython/rev/0db75a55145a

New changeset d61424122af5 by Benjamin Peterson in branch 'default':
merge 3.2 (#15846)
http://hg.python.org/cpython/rev/d61424122af5
History
Date User Action Args
2012-09-02 18:25:24python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg169717

resolution: fixed
stage: resolved
2012-09-02 09:44:58pitrousetnosy: + brett.cannon, georg.brandl, ncoghlan, benjamin.peterson
2012-09-02 09:22:21ezio.melottisettype: behavior

messages: + msg169688
nosy: + ezio.melotti
2012-09-02 08:46:06davidhaltercreate