Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fileinput should use stdin.buffer for "rb" mode #65274

Closed
brandon-rhodes mannequin opened this issue Mar 27, 2014 · 6 comments
Closed

fileinput should use stdin.buffer for "rb" mode #65274

brandon-rhodes mannequin opened this issue Mar 27, 2014 · 6 comments
Assignees
@serhiy-storchaka
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@brandon-rhodes
Copy link
Mannequin

brandon-rhodes mannequin commented Mar 27, 2014

BPO 21075
Nosy @vstinner, @serhiy-storchaka, @MojoVampire
Files
  • 21075-fileinput-stdin.diff
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-05-14.18:17:41.145>
created_at = <Date 2014-03-27.13:42:43.555>
labels = ['type-bug', 'library']
title = 'fileinput should use stdin.buffer for "rb" mode'
updated_at = <Date 2016-01-02.20:45:09.623>
user = 'https://bugs.python.org/brandon-rhodes'

    bugs.python.org fields:

    activity = <Date 2016-01-02.20:45:09.623>
actor = 'python-dev'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-05-14.18:17:41.145>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-03-27.13:42:43.555>
creator = 'brandon-rhodes'
dependencies = []
files = ['34880']
hgrepos = []
issue_num = 21075
keywords = ['patch']
message_count = 6.0
messages = ['214952', '215001', '216331', '218558', '218559', '257362']
nosy_count = 6.0
nosy_names = ['vstinner', 'brandon-rhodes', 'python-dev', 'serhiy.storchaka', 'josh.r', 'sam.kimbrel']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21075'
versions = ['Python 3.3', 'Python 3.4']
    @brandon-rhodes
    Copy link
    Mannequin Author

    brandon-rhodes mannequin commented Mar 27, 2014

    In Python 3, fileinput.input() returns str lines whether the data is
    coming from stdin or from a list of files on the command line. But if
    input(mode='rb') is specified, then its behavior becomes inconsistent:
    lines from stdin are delivered as already-decoded strings, but data
    from files is delivered (correctly) as bytes.

    The solution may be that, if a "b" is anywhere in the mode, then input()
    should read from the bytes stdin.buffer data source instead of from
    stdin.

    Otherwise the "rb" mode is rather useless since you can wind
    up getting text from it anyway depending on how you are invoked.

    @brandon-rhodes brandon-rhodes mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 27, 2014
    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Mar 27, 2014

    There is a similar, (unfixed?) bug, bpo-14156, in argparse as well. Seems like a common failing in the move to Python 3; std*.buffer was introduced, but none of the places that use it were updated, so they all became str only.

    @samkimbrel
    Copy link
    Mannequin

    samkimbrel mannequin commented Apr 15, 2014

    Patch attached that checks for 'b' in self._mode and sets self._file to sys.stdin.buffer as appropriate.

    @serhiy-storchaka serhiy-storchaka self-assigned this May 14, 2014
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 14, 2014

    New changeset 7e640fefc9c1 by Serhiy Storchaka in branch '3.4':
    Issue bpo-21075: fileinput.FileInput now reads bytes from standard stream if
    http://hg.python.org/cpython/rev/7e640fefc9c1

    New changeset 4041d4077a85 by Serhiy Storchaka in branch 'default':
    Issue bpo-21075: fileinput.FileInput now reads bytes from standard stream if
    http://hg.python.org/cpython/rev/4041d4077a85

    @serhiy-storchaka
    Copy link
    Member

    Thank you Sam for your contribution.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jan 2, 2016

    New changeset ded1336bff49 by R David Murray in branch '3.5':
    bpo-22709: Use stdin as-is if it does not have a buffer attribute.
    https://hg.python.org/cpython/rev/ded1336bff49

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @serhiy-storchaka