Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fileinput.hook_encoded has no way to pass arguments to codecs #69974

Closed
lac mannequin opened this issue Dec 3, 2015 · 15 comments
Closed

fileinput.hook_encoded has no way to pass arguments to codecs #69974

lac mannequin opened this issue Dec 3, 2015 · 15 comments
Assignees
@serhiy-storchaka
Labels
easy stdlib Python modules in the Lib dir topic-IO type-feature A feature request or enhancement

Comments

@lac
Copy link
Mannequin

lac mannequin commented Dec 3, 2015

BPO 25788
Nosy @terryjreedy, @bitdancer, @berkerpeksag, @serhiy-storchaka, @jhasapp
Files
  • issue25788.patch: Patch File
  • issue25788-2.patch
  • issue25788-3.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-04-27.20:16:43.993>
created_at = <Date 2015-12-03.16:41:30.713>
labels = ['easy', 'type-feature', 'library', 'expert-IO']
title = 'fileinput.hook_encoded has no way to pass arguments to codecs'
updated_at = <Date 2016-04-27.20:16:43.991>
user = 'https://bugs.python.org/lac'

    bugs.python.org fields:

    activity = <Date 2016-04-27.20:16:43.991>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-04-27.20:16:43.993>
closer = 'serhiy.storchaka'
components = ['Library (Lib)', 'IO']
creation = <Date 2015-12-03.16:41:30.713>
creator = 'lac'
dependencies = []
files = ['41777', '42620', '42631']
hgrepos = []
issue_num = 25788
keywords = ['patch', 'easy']
message_count = 15.0
messages = ['255818', '255819', '255824', '255826', '255827', '256164', '258072', '259347', '264179', '264193', '264334', '264351', '264379', '264400', '264401']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'r.david.murray', 'python-dev', 'berker.peksag', 'serhiy.storchaka', 'lac', 'curioswati', 'joseph.hackman']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue25788'
versions = ['Python 3.6']
    @lac
    Copy link
    Mannequin Author

    lac mannequin commented Dec 3, 2015

    Right now there is no way, aside from writing your own openhook, to
    get around the limitation that openhook=fileinput.hook_encoded("utf")
    will open things with the default option for codecs.open()
    of errors=strict. Adding a way to pass the errors argument seems
    both easy to do and useful.

    @lac lac mannequin added the topic-IO label Dec 3, 2015
    @serhiy-storchaka
    Copy link
    Member

    It would be easy to add the errors parameter to fileinput.hook_encoded().

    Do you want to provide a patch Laura?

    @serhiy-storchaka serhiy-storchaka added easy stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Dec 3, 2015
    @lac
    Copy link
    Mannequin Author

    lac mannequin commented Dec 3, 2015

    I haven't made a patch to Python for over 10 years. Before mercurial. :) Where do you start in terms of 'how to submit a patch'?

    @lac lac mannequin removed stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Dec 3, 2015
    @bitdancer
    Copy link
    Member

    Mercurial didn't change the patch submission process (we're talking about making such changes on python-workflow now...or rather enhancements as the current process should continue to work). The only thing that mercurial changed is how you get a copy of the current repo. After that (which is explained in the devguide), just make your changes, do 'hg diff', and attach the diff file to the issue as usual.

    @serhiy-storchaka
    Copy link
    Member

    https://docs.python.org/devguide/#contributing

    @serhiy-storchaka serhiy-storchaka added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Dec 3, 2015
    @curioswati
    Copy link
    Mannequin

    curioswati mannequin commented Dec 10, 2015

    I want to work on this issue. @lac, can you please help as I searched but couldn't find the related files. Where can I find the code for this?

    @terryjreedy
    Copy link
    Member

    fileinput is in Python, hence the source is in Lib. Online version is
    https://hg.python.org/cpython/file/tip/Lib/fileinput.py
    https://hg.python.org/cpython/file/tip/Lib/test/test_fileinput.py

    Patching openhook itself looks trivial. A test is the hard part. Laura, can you provide a test case? -- a short (one line?) file that fails with errors=strict and passes with something else? A new test_errors method should be added to class Test_hook_encoded (the last testCase in the file). I think the 'test' method would need revision.

    @jhasapp
    Copy link
    Mannequin

    jhasapp mannequin commented Feb 2, 2016

    I haven't seen OP in over 30 days, so am posting my own patch.

    I've added an optional argument that defaults to strict and gets passed along.

    I've updated the primary test to verify the argument passing, as well as that things get handled as specified in the documentation at https://docs.python.org/3.5/library/codecs.html

    This is off-topic, but is there any way I can submit a patch that allows a similar fix for stdin? Presently there is no way at all to pass malformed unicode through fileinput using stdin that I can find.

    @jhasapp
    Copy link
    Mannequin

    jhasapp mannequin commented Apr 25, 2016

    Ping.

    Just wondering if anyone on the nosy list would be willing to help review my patch. :)

    @serhiy-storchaka
    Copy link
    Member

    Added comments on Rietveld (follow the "review" link beside the patch link).

    @serhiy-storchaka serhiy-storchaka self-assigned this Apr 25, 2016
    @jhasapp
    Copy link
    Mannequin

    jhasapp mannequin commented Apr 27, 2016

    Uploading a new patch to address the issues in previous patch.

    @serhiy-storchaka
    Copy link
    Member

    Oh, I forget. Needed updates of the documentation (including the "versionchanged" directive). And would be nice if you add corresponding entities in Doc/whatsnew/3.6.rst, Misc/NEWS and Misc/ACKS. The rest of the patch LGTM.

    @jhasapp
    Copy link
    Mannequin

    jhasapp mannequin commented Apr 27, 2016

    Updated documentation in fileinput.rst, Doc/whatsnew/3.6.rst, Misc/NEWS and Misc/ACKS.

    Thank you so much Serhiy for taking the time to review!

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 27, 2016

    New changeset 8ab8f5259f09 by Serhiy Storchaka in branch 'default':
    Issue bpo-25788: fileinput.hook_encoded() now supports an "errors" argument
    https://hg.python.org/cpython/rev/8ab8f5259f09

    @serhiy-storchaka
    Copy link
    Member

    Committed with some changes. Thank you for your contribution Joseph.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    easy stdlib Python modules in the Lib dir topic-IO type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @bitdancer @serhiy-storchaka @terryjreedy