classification
Title: lib2to3/tests/pytree_idempotency.py has broken imports
Type: behavior Stage: resolved
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: benjamin.peterson, imz, python-dev, terry.reedy
Priority: normal Keywords:

Created on 2016-05-02 19:45 by imz, last changed 2016-05-06 21:56 by terry.reedy. This issue is now closed.

Messages (5)
msg264663 - (view) Author: Ivan Zakharyaschev (imz) Date: 2016-05-02 19:45
Line 21 of lib2to3/tests/pytree_idempotency.py:

import pgen2

This seems to be broken, because it should be:

import lib2to3.pgen2

(This makes the auto-requirements generator in ALT Sisyphus to generate a broken/unsatisfiable dependency on pgen2 when packaging this into an RPM.)

All other uses of pgen2 in Python 3.5.1 sources use other forms of import: either relative or the absolute one as I suggest:

$ git grep -F pgen2
lib2to3/btm_matcher.py:        # from .pgen2 import token // token.__dict__.items():
lib2to3/btm_utils.py:from .pgen2 import grammar, token
lib2to3/fixer_util.py:from .pgen2 import token
lib2to3/fixes/fix_apply.py:from ..pgen2 import token
lib2to3/fixes/fix_dict.py:from ..pgen2 import token
lib2to3/fixes/fix_except.py:from ..pgen2 import token
lib2to3/fixes/fix_filter.py:from ..pgen2 import token
lib2to3/fixes/fix_has_key.py:from ..pgen2 import token
lib2to3/fixes/fix_map.py:from ..pgen2 import token
lib2to3/fixes/fix_ne.py:from ..pgen2 import token
lib2to3/fixes/fix_next.py:from ..pgen2 import token
lib2to3/fixes/fix_numliterals.py:from ..pgen2 import token
lib2to3/fixes/fix_print.py:from ..pgen2 import token
lib2to3/fixes/fix_raise.py:from ..pgen2 import token
lib2to3/fixes/fix_throw.py:from ..pgen2 import token
lib2to3/fixes/fix_tuple_params.py:from ..pgen2 import token
lib2to3/fixes/fix_types.py:from ..pgen2 import token
lib2to3/fixes/fix_unicode.py:from ..pgen2 import token
lib2to3/fixes/fix_ws_comma.py:from ..pgen2 import token
lib2to3/patcomp.py:from .pgen2 import driver, literals, token, tokenize, parse, grammar
lib2to3/pgen2/__init__.py:"""The pgen2 package."""
lib2to3/pgen2/conv.py:from pgen2 import grammar, token
lib2to3/pgen2/token.py:#   originally monkeypatched in by pgen2.tokenize
lib2to3/pgen2/tokenize.py:from lib2to3.pgen2.token import *
lib2to3/pygram.py:from .pgen2 import token
lib2to3/pygram.py:from .pgen2 import driver
lib2to3/pytree.py:        # from .pgen2 import token // token.__dict__.items():
lib2to3/refactor.py:from .pgen2 import driver, tokenize, token
lib2to3/tests/pytree_idempotency.py:import pgen2
lib2to3/tests/pytree_idempotency.py:from pgen2 import driver
lib2to3/tests/pytree_idempotency.py:            except pgen2.parse.ParseError as err:
lib2to3/tests/support.py:from lib2to3.pgen2 import driver
lib2to3/tests/test_parser.py:from lib2to3.pgen2 import tokenize
lib2to3/tests/test_parser.py:from ..pgen2.parse import ParseError
lib2to3/tests/test_refactor.py:from lib2to3.pgen2 import token
lib2to3/tests/test_util.py:from lib2to3.pgen2 import token
msg264665 - (view) Author: Ivan Zakharyaschev (imz) Date: 2016-05-02 19:47
BTW, this source file seems to be not used anywhere.
msg265015 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-05-06 19:58
New changeset 3279c910d0e0 by Terry Jan Reedy in branch '3.5':
Issue 26911: fix import (other problems remain).
https://hg.python.org/cpython/rev/3279c910d0e0
msg265026 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-05-06 20:38
(Import errors are runtime errors, not compile errors.)

After 'import lib2to3.pgen2', 'pgen2' in the code would not be valid.  A 'from' import is needed instead.  I fixed the import for 3.5/6 (but not 2.7), but the file still does not run.

  File "F:\Python\dev\35\lib\lib2to3\tests\pytree_idempotency.py", line 27, in main
    gr = driver.load_grammar("Grammar.txt")
 ,,,
FileNotFoundError: [Errno 2] No such file or directory: 'Grammar.txt'

After fixing this with

 def main():
-    gr = driver.load_grammar("Grammar.txt")
+    gfile = os.path.dirname(os.path.dirname(__file__)) + "/Grammar.txt"
+    gr = driver.load_grammar(gfile)
     dr = driver.Driver(gr, convert=pytree.convert)
 
the file fails with

FileNotFoundError: [Errno 2] No such file or directory: 'example.py'

There is no 'example.py in lib2to3.  The closest is lib2to3/tests/data/fixers/parrot_example.py.  At this point, I leave it Benjamin to fix the file so it runs or delete it.

The file is not used anywhere since it is not part of the regular test suite (that would require the name to begin with 'test_').  It was meant to be run as a test by itself -- see the doc string.  Perhaps the completion of 2to3 and the test suite made it obsolete.
msg265036 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-05-06 21:56
I decided that I should leave the scope of this issue as it was and close it as fixed.
History
Date User Action Args
2016-05-06 21:56:12terry.reedysetstatus: open -> closed

assignee: terry.reedy
title: lib2to3/tests/pytree_idempotency.py does not run -> lib2to3/tests/pytree_idempotency.py has broken imports
resolution: fixed
versions: - Python 2.7
messages: + msg265036
stage: needs patch -> resolved
2016-05-06 20:38:16terry.reedysetversions: + Python 2.7, Python 3.6
type: compile error -> behavior

nosy: + terry.reedy
title: lib2to3/tests/pytree_idempotency.py has a broken import -> lib2to3/tests/pytree_idempotency.py does not run
messages: + msg265026
stage: needs patch
2016-05-06 19:58:37python-devsetnosy: + python-dev
messages: + msg265015
2016-05-02 19:51:12serhiy.storchakasetnosy: + benjamin.peterson
2016-05-02 19:47:44imzsetmessages: + msg264665
2016-05-02 19:45:26imzcreate