classification
Title: AssertionError in lib2to3 on 2.7.11 Windows
Type: crash Stage: resolved
Components: 2to3 (2.x to 3.x conversion tool), Windows Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: paul.moore, schlamar, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2015-12-21 10:02 by schlamar, last changed 2017-06-18 18:24 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
PatternGrammar2.7.11.final.0.pickle schlamar, 2015-12-21 10:05
Grammar2.7.11.final.0.pickle schlamar, 2015-12-21 10:05
Messages (7)
msg256790 - (view) Author: Marc Schlaich (schlamar) * Date: 2015-12-21 10:02
This simple test case results in a lib2to3 crash with an AssertionError in 2.7.11


source = '''

class Test(object):

    tests = []


'''

from lib2to3.refactor import RefactoringTool


def main():
    tool = RefactoringTool([])
    tool.refactor_string(source, '')

        
Exception:

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    main()
  File "test.py", line 16, in main
    tool.refactor_string(source, '')
  File "c:\Python27\Lib\lib2to3\refactor.py", line 380, in refactor_string
    name, err.__class__.__name__, err)
  File "c:\Python27\Lib\lib2to3\refactor.py", line 377, in refactor_string
    tree = self.driver.parse_string(data)
  File "c:\Python27\Lib\lib2to3\pgen2\driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "c:\Python27\Lib\lib2to3\pgen2\driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "c:\Python27\Lib\lib2to3\pgen2\parse.py", line 127, in addtoken
    assert t < 256
AssertionError

Pretty weird: A manually built Python 2.7.11 with VS 2008 doesn't show this issue.
msg256791 - (view) Author: Marc Schlaich (schlamar) * Date: 2015-12-21 10:04
The issue is in Grammar2.7.11.final.0.pickle or PatternGrammar2.7.11.final.0.pickle. I have copied these files to my manually built Python and getting the same issue in this case.
msg256792 - (view) Author: Marc Schlaich (schlamar) * Date: 2015-12-21 10:05
I'm attaching those bad files.
msg256793 - (view) Author: Marc Schlaich (schlamar) * Date: 2015-12-21 10:07
After deleting them, they are getting recreated and everything works now.
msg257001 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-12-25 17:49
If I understand your last comment, this issue should be closed.  On Win10, 2.7.11, I do not get the error.
msg257173 - (view) Author: Marc Schlaich (schlamar) * Date: 2015-12-29 10:53
Not sure. Somehow I got these corrupted files just by upgrading to 2.7.11. It might be that they are not getting updated when installing 2.7.11 over an existing 2.7.10. So maybe it's worth to investigate further as others could be affected, too.
msg261167 - (view) Author: Marc Schlaich (schlamar) * Date: 2016-03-03 13:45
We have some business privilege management solution running which might have corrupted the installation. As no one else is reporting this issue, this is my best guest for this phenomena and I'm going to close this issue.
History
Date User Action Args
2017-06-18 18:24:44terry.reedysetresolution: not a bug
stage: resolved
2016-03-03 13:45:38schlamarsetstatus: open -> closed

messages: + msg261167
2015-12-29 10:53:29schlamarsetmessages: + msg257173
2015-12-25 17:49:22terry.reedysetnosy: + terry.reedy
messages: + msg257001
2015-12-21 16:16:11zach.waresetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows
2015-12-21 10:07:31schlamarsetmessages: + msg256793
2015-12-21 10:05:53schlamarsetfiles: + Grammar2.7.11.final.0.pickle
2015-12-21 10:05:43schlamarsetfiles: + PatternGrammar2.7.11.final.0.pickle

messages: + msg256792
2015-12-21 10:04:07schlamarsetmessages: + msg256791
2015-12-21 10:02:07schlamarcreate