Running 2to3 on lxml-2.2.6 failed with broken indentation.

This fragment from lxml/html/__init__.py:
> def open_http_urllib(method, url, values):
>     ## FIXME: should test that it's not a relative URL or something
>     try:
>        from urllib import urlencode, urlopen
>     except ImportError: # Python 3

became:
> def open_http_urllib(method, url, values):
>     ## FIXME: should test that it's not a relative URL or something
>     try:
>         from urllib.parse import urlencode
> from urllib.request import urlopen
>     except ImportError: # Python 3

which is syntactically invalid: note the invalid indentation for the second "import" line.

Seems to work when there's a single name imported per line; fails when more than one name is imported from urlib.


Am attaching a patch to test_fixers (for the 2.7 branch) which adds a reproducer.  I don't have a fix.

test test_lib2to3 failed -- Traceback (most recent call last):
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 1858, in test_import_indentation
    self.check(b, a)
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 37, in check
    tree = self._check(before, after)
  File "/home/david/coding/python-svn/2.7-2to3/Lib/lib2to3/tests/test_fixers.py", line 33, in _check
    self.assertEqual(after, unicode(tree))
AssertionError: u"\ndef foo():\n    from urllib.parse import urlencode\n    from urllib.parse im [truncated]... != u"\ndef foo():\n    from urllib.parse import urlencode\nfrom urllib.request impo [truncated]...
  
  def foo():
      from urllib.parse import urlencode
-     from urllib.parse import urlopen
? ----            --  ^
+ from urllib.request import urlopen
?              ++++ ^
      print('got here')
  
(Note to self: tracking this downstream in Fedora for lxml as https://bugzilla.redhat.com/show_bug.cgi?id=600036 )

I guess it's the problem with lib2to3/fixes/fix_urllib.py. Indentation is not taken into consideration when fix "import". Fix it with indentation taken into consideration maybe a little complex, but here is a simple and ugly fix: when one import statement was transformed to two or more statement, use semicolon(';') instead of '\n' as separator, between many import statements. In this case, the 
"""
def fun():
    from urllib import urlopen, urlencode
"""

is transformed in to:

"""
def fun():
    from urllib.request import urlopen;from urllib.parse import urlencode
"""

It will takes time to work out a better fix.

When one import statement is split to two or more, we encounter this problem: the indentation of the import statements except the first one is unknown, and is difficult to fix this problem, since a import maybe in a multi-statement line, like: 'a=1;import sys'. I wonder if there is way to fix this problem perfectly. Maybe we could just put all the resulting import statements into one single multi-statement line, joined by ';', as my patch specified.

Fixed in r83798.