classification
Title: 2to3 does not preserve line endings
Type: behavior Stage: test needed
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: benjamin.peterson Nosy List: benjamin.peterson, bialix, eric.araujo
Priority: normal Keywords: patch

Created on 2011-03-18 16:27 by bialix, last changed 2011-06-09 15:13 by eric.araujo.

Files
File name Uploaded Description Edit
2to3-crlf-test.diff eric.araujo, 2011-06-09 15:13
Messages (7)
msg131335 - (view) Author: Alexander Belchenko (bialix) Date: 2011-03-18 16:27
I'm using LF-only line-endings for development of my IntelHex library. I'm working on Windows most of the time.

After 2to3 tool has been ran on my library it has not only changed the Python syntax, but it also saved all files with CRLF line-endings. As result I have all changed files completelly modified and diff shows change in every line. 

2to3 tool should respect my line-endings and must not use simple open(xxx, "wt") mode for writing modified files.
msg131349 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-03-18 18:19
Thanks for the report.  Can you run “python -m test.test_lib2to3”, if possible with a Python 3.x version?  I’ve seen that the tests use binary mode to compare file contents, so maybe you will get an error message that can get us started.
msg131626 - (view) Author: Alexander Belchenko (bialix) Date: 2011-03-21 09:38
@Éric Araujo: I've ran tests with python 3.2. All tests have passed:

----------------------------------------------------------------------
Ran 540 tests in 37.688s

OK
msg131639 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-03-21 12:29
Thanks.  Would you like to work on a unit test or full patch?
msg131645 - (view) Author: Alexander Belchenko (bialix) Date: 2011-03-21 13:16
Éric, thank you for the proposal, but I'm not familiar enough with the codebase to work on it. 

The short scan over the tests reveals that there is at least one test which tries to test CRLF behavior, in the file test_refactor.py, but I don't understand what it doing?

    def test_crlf_newlines(self):
        old_sep = os.linesep
        os.linesep = "\r\n"
        try:
            fn = os.path.join(TEST_DATA_DIR, "crlf.py")
            fixes = refactor.get_fixers_from_package("lib2to3.fixes")
            self.check_file_refactoring(fn, fixes)
        finally:
            os.linesep = old_sep

So, in theory I can modify that test to check what if the file has LF-only line-endings originally, but os.linesep is CRLF, but then I don't know what the content I should create and how to run fixer over that.
msg131652 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-03-21 13:53
I can fix it. I just need to find time. :)
msg137983 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-09 15:13
I was surprised to see that the crlf.py file was not using CRLF in the new Mercurial repo.  It is also not in the .hgeol file.  I changed it locally, but it doesn’t change anything, the tests pass before and after the change.
History
Date User Action Args
2011-06-09 15:13:39eric.araujosetfiles: + 2to3-crlf-test.diff
keywords: + patch
messages: + msg137983

versions: - Python 3.1
2011-03-21 13:53:32benjamin.petersonsetnosy: bialix, benjamin.peterson, eric.araujo
messages: + msg131652
2011-03-21 13:16:34bialixsetnosy: bialix, benjamin.peterson, eric.araujo
messages: + msg131645
2011-03-21 12:29:29eric.araujosetnosy: bialix, benjamin.peterson, eric.araujo
title: 2to3 tool does not preserve line-endings -> 2to3 does not preserve line endings
messages: + msg131639
stage: test needed
2011-03-21 09:38:15bialixsetnosy: bialix, benjamin.peterson, eric.araujo
messages: + msg131626
2011-03-18 23:51:15benjamin.petersonsetassignee: benjamin.peterson
nosy: bialix, benjamin.peterson, eric.araujo
2011-03-18 18:19:16eric.araujosetnosy: + eric.araujo

messages: + msg131349
versions: + Python 3.1, Python 3.3
2011-03-18 17:18:59r.david.murraysetnosy: + benjamin.peterson
2011-03-18 16:27:27bialixcreate