classification
Title: codecs.open() + eol (windows)
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: lemburg Nosy List: amaury.forgeotdarc, analyst, lemburg, shamilbi
Priority: normal Keywords: easy, patch

Created on 2009-11-04 14:13 by shamilbi, last changed 2014-03-09 06:23 by analyst.

Files
File name Uploaded Description Edit
eol-bug.py shamilbi, 2009-11-04 14:13 code to reproduce
7262.patch analyst, 2014-03-09 06:23 Patch review
Messages (5)
msg94888 - (view) Author: (shamilbi) Date: 2009-11-04 14:13
different eol when writing to fp = codecs.open(.., 'w', 'cp866')
(windows, python-2.6.4)

def write(fp):
    fp.write("""\
a
""")

# eol=0d0a (windows, python-2.6.4)
with open('0d0a.tmp', 'w') as fp:
    write(fp)

# eol=0d0a (windows, python-2.6.4)
with codecs.open('0d0a-codecs.tmp', 'w') as fp:
    write(fp)

# --- BUG ---
# eol=0a (windows, python-2.6.4)
with codecs.open('0a-codecs.tmp', 'w', 'cp866') as fp:
    write(fp)
msg94895 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-11-04 18:02
The docs say::
  Files are always opened in binary mode, even if no binary mode was
  specified. This is done to avoid data loss due to encodings using
  8-bit values. This means that no automatic conversion of '\n' is done
  on reading and writing.

But this does not match the code of codecs.open()::
    if encoding is not None and \
       'b' not in mode:
        # Force opening of the file in binary mode
        mode = mode + 'b'

When the encoding is None, the file is opened in text mode.
Marc-Andre, what do you think? is it a documentation bug instead?
msg94898 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2009-11-04 18:18
Amaury Forgeot d'Arc wrote:
> 
> Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
> 
> The docs say::
>   Files are always opened in binary mode, even if no binary mode was
>   specified. This is done to avoid data loss due to encodings using
>   8-bit values. This means that no automatic conversion of '\n' is done
>   on reading and writing.
> 
> But this does not match the code of codecs.open()::
>     if encoding is not None and \
>        'b' not in mode:
>         # Force opening of the file in binary mode
>         mode = mode + 'b'
> 
> When the encoding is None, the file is opened in text mode.
> Marc-Andre, what do you think? is it a documentation bug instead?

Agreed, it's a documentation bug. If no encoding is specified,
codecs.open() works just like the standard open() (except for
the default value of mode, but that's documented).

The idea was to provide a drop-in replacement for open() - with
the added encoding parameter support.

Perhaps the default mode should have been 'r' to match the
regular open() - I guess it's too late to change that, though.
msg189621 - (view) Author: Mark Lawrence (BreamoreBoy) Date: 2013-05-19 19:56
The docs still read the same.  Would someone in the know like to propose a doc patch please.
msg212957 - (view) Author: (analyst) Date: 2014-03-09 06:23
Hi,
  I am new to Python Development. I would like to propose a patch for this issue.
History
Date User Action Args
2014-03-09 06:23:26analystsetfiles: + 7262.patch

nosy: + analyst
messages: + msg212957

keywords: + patch
2014-02-03 15:34:47BreamoreBoysetnosy: - BreamoreBoy
2014-01-29 07:44:20serhiy.storchakasetkeywords: + easy
stage: needs patch
components: + Documentation, - Library (Lib)
versions: + Python 2.7, Python 3.3, Python 3.4, - Python 2.6
2013-05-19 19:56:34BreamoreBoysetnosy: + BreamoreBoy
messages: + msg189621
2009-11-04 18:18:06lemburgsetmessages: + msg94898
2009-11-04 18:02:07amaury.forgeotdarcsetassignee: lemburg

messages: + msg94895
nosy: + amaury.forgeotdarc, lemburg
2009-11-04 14:13:29shamilbicreate