Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[doc] codecs.open() + eol (windows) #51511

Closed
shamilbi mannequin opened this issue Nov 4, 2009 · 7 comments
Closed

[doc] codecs.open() + eol (windows) #51511

shamilbi mannequin opened this issue Nov 4, 2009 · 7 comments
Assignees
@malemburg
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir easy type-bug An unexpected behavior, bug, or error

Comments

@shamilbi
Copy link
Mannequin

shamilbi mannequin commented Nov 4, 2009

BPO 7262
Nosy @malemburg, @amauryfa, @iritkatriel, @kumaraditya303
PRs
  • bpo-26897: Clarify Popen stdin, stdout, stderr file object docs #30231
    		•
    Files
  • eol-bug.py: code to reproduce
  • 7262.patch: Patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/malemburg'
closed_at = None
created_at = <Date 2009-11-04.14:13:29.252>
labels = ['easy', 'type-bug', '3.9', '3.10', '3.11', 'docs']
title = '[doc] codecs.open() + eol (windows)'
updated_at = <Date 2021-12-22.12:25:06.153>
user = 'https://bugs.python.org/shamilbi'

    bugs.python.org fields:

    activity = <Date 2021-12-22.12:25:06.153>
actor = 'kumaraditya'
assignee = 'lemburg'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2009-11-04.14:13:29.252>
creator = 'shamilbi'
dependencies = []
files = ['15256', '34309']
hgrepos = []
issue_num = 7262
keywords = ['patch', 'easy']
message_count = 6.0
messages = ['94888', '94895', '94898', '189621', '212957', '408163']
nosy_count = 6.0
nosy_names = ['lemburg', 'amaury.forgeotdarc', 'shamilbi', 'analyst', 'iritkatriel', 'kumaraditya']
pr_nums = ['30231']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue7262'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
    @shamilbi
    Copy link
    Mannequin Author

    shamilbi mannequin commented Nov 4, 2009

    different eol when writing to fp = codecs.open(.., 'w', 'cp866')
    (windows, python-2.6.4)

    def write(fp):
    fp.write("""\
a
""")

# eol=0d0a (windows, python-2.6.4)
with open('0d0a.tmp', 'w') as fp:
    write(fp)

# eol=0d0a (windows, python-2.6.4)
with codecs.open('0d0a-codecs.tmp', 'w') as fp:
    write(fp)

    # --- BUG ---

    # eol=0a (windows, python-2.6.4)
with codecs.open('0a-codecs.tmp', 'w', 'cp866') as fp:
    write(fp)

    @shamilbi shamilbi mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Nov 4, 2009
    @amauryfa
    Copy link
    Member

    amauryfa commented Nov 4, 2009

    The docs say::
    Files are always opened in binary mode, even if no binary mode was
    specified. This is done to avoid data loss due to encodings using
    8-bit values. This means that no automatic conversion of '\n' is done
    on reading and writing.

    But this does not match the code of codecs.open()::
    if encoding is not None and \
    'b' not in mode:
    # Force opening of the file in binary mode
    mode = mode + 'b'

    When the encoding is None, the file is opened in text mode.
    Marc-Andre, what do you think? is it a documentation bug instead?

    @malemburg
    Copy link
    Member

    Amaury Forgeot d'Arc wrote:

    Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:

    The docs say::
    Files are always opened in binary mode, even if no binary mode was
    specified. This is done to avoid data loss due to encodings using
    8-bit values. This means that no automatic conversion of '\n' is done
    on reading and writing.

    But this does not match the code of codecs.open()::
    if encoding is not None and \
    'b' not in mode:
    # Force opening of the file in binary mode
    mode = mode + 'b'

    When the encoding is None, the file is opened in text mode.
    Marc-Andre, what do you think? is it a documentation bug instead?

    Agreed, it's a documentation bug. If no encoding is specified,
    codecs.open() works just like the standard open() (except for
    the default value of mode, but that's documented).

    The idea was to provide a drop-in replacement for open() - with
    the added encoding parameter support.

    Perhaps the default mode should have been 'r' to match the
    regular open() - I guess it's too late to change that, though.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented May 19, 2013

    The docs still read the same. Would someone in the know like to propose a doc patch please.

    @serhiy-storchaka serhiy-storchaka added easy docs Documentation in the Doc dir and removed stdlib Python modules in the Lib dir labels Jan 29, 2014
    @analyst
    Copy link
    Mannequin

    analyst mannequin commented Mar 9, 2014

    Hi,
    I am new to Python Development. I would like to propose a patch for this issue.

    @iritkatriel
    Copy link
    Member

    That paragraph was edited here:

    b9fdb7a

    but this point was not added.

    @iritkatriel iritkatriel added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Dec 9, 2021
    @iritkatriel iritkatriel changed the title codecs.open() + eol (windows) [doc] codecs.open() + eol (windows) Dec 9, 2021
    @kumaraditya303 kumaraditya303 added 3.9 only security fixes 3.10 only security fixes and removed 3.9 only security fixes 3.10 only security fixes labels Dec 22, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @slateny slateny mentioned this issue Jun 28, 2022
    Merged
    gvanrossum pushed a commit that referenced this issue Oct 21, 2022
    @slateny
    gh-51511: Note that codecs.open()'s encoding parameter affects automa…
    d9407b1 
    …tic conversion to binary mode (#94370)
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 21, 2022
    @slateny @miss-islington
    pythongh-51511: Note that codecs.open()'s encoding parameter affects …
    5e0e443 
    …automatic conversion to binary mode (pythonGH-94370)

(cherry picked from commit d9407b1)

Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
    @bedevere-bot bedevere-bot mentioned this issue Oct 21, 2022
    Merged
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 21, 2022
    @slateny @miss-islington
    pythongh-51511: Note that codecs.open()'s encoding parameter affects …
    5bb8746 
    …automatic conversion to binary mode (pythonGH-94370)

(cherry picked from commit d9407b1)

Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
    @bedevere-bot bedevere-bot mentioned this issue Oct 21, 2022
    Merged
    miss-islington added a commit that referenced this issue Oct 21, 2022
    @miss-islington @slateny
    gh-51511: Note that codecs.open()'s encoding parameter affects automa…
    c835b97 
    …tic conversion to binary mode (GH-94370)

(cherry picked from commit d9407b1)

Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
    miss-islington added a commit that referenced this issue Oct 21, 2022
    @miss-islington @slateny
    gh-51511: Note that codecs.open()'s encoding parameter affects automa…
    dcb342b 
    …tic conversion to binary mode (GH-94370)

(cherry picked from commit d9407b1)

Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
    @hauntsaninja
    Copy link
    Contributor

    Thanks, looks like this was completed

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @malemburg malemburg

    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir easy type-bug An unexpected behavior, bug, or error
    Projects
    Codecs and encodings issues
    Status: Done
    Milestone
    No milestone
    Development

    No branches or pull requests
    6 participants
    @amauryfa @iritkatriel @serhiy-storchaka @malemburg @hauntsaninja @kumaraditya303