classification
Title: Bad indentation in urllib import fixer with multiple imports
Type: behavior Stage: needs patch
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: benjamin.peterson Nosy List: benjamin.peterson, dmalcolm, eric.araujo, ysj.ray
Priority: normal Keywords: patch

Created on 2010-07-26 17:28 by dmalcolm, last changed 2010-08-07 23:55 by benjamin.peterson. This issue is now closed.

Files
File name Uploaded Description Edit
urllib-indentation-issue.patch dmalcolm, 2010-07-26 17:28
lib2to3.diff ysj.ray, 2010-07-28 11:14 patch against the trunk.
Messages (4)
msg111649 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-07-26 17:28
Running 2to3 on lxml-2.2.6 failed with broken indentation.

This fragment from lxml/html/__init__.py:
> def open_http_urllib(method, url, values):
>     ## FIXME: should test that it's not a relative URL or something
>     try:
>        from urllib import urlencode, urlopen
>     except ImportError: # Python 3

became:
> def open_http_urllib(method, url, values):
>     ## FIXME: should test that it's not a relative URL or something
>     try:
>         from urllib.parse import urlencode
> from urllib.request import urlopen
>     except ImportError: # Python 3

which is syntactically invalid: note the invalid indentation for the second "import" line.

Seems to work when there's a single name imported per line; fails when more than one name is imported from urlib.


Am attaching a patch to test_fixers (for the 2.7 branch) which adds a reproducer.  I don't have a fix.

test test_lib2to3 failed -- Traceback (most recent call last):
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 1858, in test_import_indentation
    self.check(b, a)
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 37, in check
    tree = self._check(before, after)
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 33, in _check
    self.assertEqual(after, unicode(tree))
AssertionError: u"\ndef foo():\n    from urllib.parse import urlencode\n    from urllib.parse im [truncated]... != u"\ndef foo():\n    from urllib.parse import urlencode\nfrom urllib.request impo [truncated]...
  
  def foo():
      from urllib.parse import urlencode
-     from urllib.parse import urlopen
? ----            --  ^
+ from urllib.request import urlopen
?              ++++ ^
      print('got here')
  
(Note to self: tracking this downstream in Fedora for lxml as https://bugzilla.redhat.com/show_bug.cgi?id=600036 )
msg111788 - (view) Author: ysj.ray (ysj.ray) Date: 2010-07-28 11:14
I guess it's the problem with lib2to3/fixes/fix_urllib.py. Indentation is not taken into consideration when fix "import". Fix it with indentation taken into consideration maybe a little complex, but here is a simple and ugly fix: when one import statement was transformed to two or more statement, use semicolon(';') instead of '\n' as separator, between many import statements. In this case, the 
"""
def fun():
    from urllib import urlopen, urlencode
"""

is transformed in to:

"""
def fun():
    from urllib.request import urlopen;from urllib.parse import urlencode
"""

It will takes time to work out a better fix.
msg111794 - (view) Author: ysj.ray (ysj.ray) Date: 2010-07-28 12:18
When one import statement is split to two or more, we encounter this problem: the indentation of the import statements except the first one is unknown, and is difficult to fix this problem, since a import maybe in a multi-statement line, like: 'a=1;import sys'. I wonder if there is way to fix this problem perfectly. Maybe we could just put all the resulting import statements into one single multi-statement line, joined by ';', as my patch specified.
msg113219 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-08-07 23:55
Fixed in r83798.
History
Date User Action Args
2010-08-07 23:55:39benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg113219
2010-07-29 14:50:08georg.brandlsetassignee: benjamin.peterson
2010-07-28 12:18:19ysj.raysetmessages: + msg111794
2010-07-28 11:14:23ysj.raysetfiles: + lib2to3.diff
nosy: + ysj.ray
messages: + msg111788

2010-07-28 09:33:44eric.araujosetnosy: + benjamin.peterson, eric.araujo
2010-07-26 17:28:39dmalcolmsettitle: Bad indentation in urllib import fixer with multipe imports -> Bad indentation in urllib import fixer with multiple imports
2010-07-26 17:28:23dmalcolmcreate