Created on 2017-04-20 12:45 by haypo, last changed 2017-04-21 16:14 by Eric Appelt.
|PR 1242||open||Eric Appelt, 2017-04-21 16:13|
|msg291970 - (view)||Author: STINNER Victor (haypo) *||Date: 2017-04-20 12:45|
The test_lib2to3.test_parser.test_all_project_files() test fails and produces an annoying output. See the issue #13125 Example of a buildbot: 0:11:09 [398/404] test_lib2to3 passed (39 sec) --- /root/buildarea/3.x.angelico-debian-amd64/build/Lib/lib2to3/tests/data/bom.py 2017-02-11 12:20:43.532000000 +1100 +++ @ 2017-04-20 22:05:37.157911808 +1000 @@ -1,2 +1,2 @@ -# coding: utf-8 +# coding: utf-8 print "BOM BOOM!" Example by running: ./python -m test -v test_lib2to3 --- test_all_project_files (lib2to3.tests.test_parser.TestParserIdempotency) ... /home/haypo/prog/python/master/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/haypo/prog/python/master/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38))) warnings.warn('ParseError on file %s (%s)' % (filepath, err)) /home/haypo/prog/python/master/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/haypo/prog/python/master/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33))) warnings.warn('ParseError on file %s (%s)' % (filepath, err)) --- /home/haypo/prog/python/master/Lib/lib2to3/tests/data/bom.py 2017-02-10 23:10:03.392778645 +0100 +++ @ 2017-04-20 14:32:49.921613096 +0200 @@ -1,2 +1,2 @@ -# coding: utf-8 +# coding: utf-8 print "BOM BOOM!" expected failure --- The test fails to parse the following code: --- from __future__ import print_function import sys print("WARNING", file=sys.stderr) --- whereas lib2to3 is able to parse the code. It seems like the code uses a lib2to3.tests.support.driver object. Maybe this object lacks the fixer which parses the __future__ imports? The minimum fix would be to make the test quiet: remove the warning, don't log anything. The best would be fix the test.
|msg292059 - (view)||Author: Eric Appelt (Eric Appelt) *||Date: 2017-04-21 16:14|
I added a PR to fix these two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test with the following changes to the test: 1. Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig. 2. If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM. For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change.
|2017-04-21 16:14:58||Eric Appelt||set||nosy:
+ Eric Appelt|
messages: + msg292059
|2017-04-21 16:13:03||Eric Appelt||set||pull_requests: + pull_request1360|