This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author jaraco
Recipients Arfrever, gumblex, jaraco, terry.reedy
Date 2015-06-21.00:14:42
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1434845684.29.0.156440000424.issue20387@psf.upfronthosting.co.za>
In-reply-to
Content
I've committed the patch without the change for "at least two elements" as https://bitbucket.org/jaraco/cpython-issue20387/commits/b7fe3c865b8dbdb33d26f4bc5cbb6096f5445fb2.

The patch corrects the new test, demonstrating its effectiveness, but yields two new test failures (apparent regressions):

$ ./python.exe Lib/test/test_tokenize.py 
**********************************************************************
File "/public/cpython-issue20387/Lib/test/test_tokenize.py", line ?, in test.test_tokenize.__test__.doctests
Failed example:
    roundtrip("try: import somemodule\n"
              "except ImportError: # comment\n"
              "    print('Can not import' # comment2\n)"
              "else:   print('Loaded')\n")
Exception raised:
    Traceback (most recent call last):
      File "/public/cpython-issue20387/Lib/doctest.py", line 1318, in __run
        compileflags, 1), test.globs)
      File "<doctest test.test_tokenize.__test__.doctests[14]>", line 1, in <module>
        roundtrip("try: import somemodule\n"
      File "/public/cpython-issue20387/Lib/test/test_tokenize.py", line 698, in roundtrip
        bytes_from5 = untokenize(tokens5)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 339, in untokenize
        out = ut.untokenize(iterable)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 273, in untokenize
        self.add_whitespace(start)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 236, in add_whitespace
        .format(row, col, self.prev_row, self.prev_col))
    ValueError: start (4,1) precedes previous end (4,4)
**********************************************************************
File "/public/cpython-issue20387/Lib/test/test_tokenize.py", line ?, in test.test_tokenize.__test__.doctests
Failed example:
    for testfile in testfiles:
        if not roundtrip(open(testfile, 'rb')):
            print("Roundtrip failed for file %s" % testfile)
            break
    else: True
Exception raised:
    Traceback (most recent call last):
      File "/public/cpython-issue20387/Lib/doctest.py", line 1318, in __run
        compileflags, 1), test.globs)
      File "<doctest test.test_tokenize.__test__.doctests[70]>", line 2, in <module>
        if not roundtrip(open(testfile, 'rb')):
      File "/public/cpython-issue20387/Lib/test/test_tokenize.py", line 698, in roundtrip
        bytes_from5 = untokenize(tokens5)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 339, in untokenize
        out = ut.untokenize(iterable)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 273, in untokenize
        self.add_whitespace(start)
      File "/public/cpython-issue20387/Lib/tokenize.py", line 236, in add_whitespace
        .format(row, col, self.prev_row, self.prev_col))
    ValueError: start (73,8) precedes previous end (73,12)
**********************************************************************
1 items had failures:
   2 of  74 in test.test_tokenize.__test__.doctests
***Test Failed*** 2 failures.
Traceback (most recent call last):
  File "Lib/test/test_tokenize.py", line 1261, in <module>
    test_main()
  File "Lib/test/test_tokenize.py", line 1252, in test_main
    support.run_doctest(test_tokenize, True)
  File "/public/cpython-issue20387/Lib/test/support/__init__.py", line 1854, in run_doctest
    raise TestFailed("%d of %d doctests failed" % (f, t))
test.support.TestFailed: 2 of 79 doctests failed

@gumblex, Can you review the failures and address them?
History
Date User Action Args
2015-06-21 00:14:45jaracosetrecipients: + jaraco, terry.reedy, Arfrever, gumblex
2015-06-21 00:14:44jaracosetmessageid: <1434845684.29.0.156440000424.issue20387@psf.upfronthosting.co.za>
2015-06-21 00:14:44jaracolinkissue20387 messages
2015-06-21 00:14:42jaracocreate