Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accept mutable bytes-like objects #67190

Closed
serhiy-storchaka opened this issue Dec 6, 2014 · 14 comments
Closed

Accept mutable bytes-like objects #67190

serhiy-storchaka opened this issue Dec 6, 2014 · 14 comments
Assignees
@serhiy-storchaka
Labels
extension-modules C modules in the Modules dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

BPO 23001
Nosy @pitrou, @vstinner, @ethanfurman, @vadmium, @serhiy-storchaka, @MojoVampire
Files
  • accept_mutable_buffers.patch
  • accept_mutable_buffers_2.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-03-20.07:06:26.080>
created_at = <Date 2014-12-06.14:25:06.583>
labels = ['extension-modules', 'type-feature']
title = 'Accept mutable bytes-like objects'
updated_at = <Date 2015-03-20.10:01:21.895>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2015-03-20.10:01:21.895>
actor = 'vstinner'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-03-20.07:06:26.080>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2014-12-06.14:25:06.583>
creator = 'serhiy.storchaka'
dependencies = []
files = ['37373', '38555']
hgrepos = []
issue_num = 23001
keywords = ['patch', 'needs review']
message_count = 14.0
messages = ['232244', '232246', '232252', '232255', '232264', '233438', '238502', '238598', '238607', '238608', '238628', '238630', '238633', '238635']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'Arfrever', 'ethan.furman', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'josh.r']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23001'
versions = ['Python 3.5']
    @serhiy-storchaka
    Copy link
    Member Author

    Some builtins accept only read-only bytes-like objects (PyArg_Parse format codes "s#", "z#", "y", and "y#"). Proposed patch makes them accepting also mutable bytes-like objects such as bytearray.

    I'm not sure that all these changes are useful, but in some cases this can get rid of copying binary data created in bytearray (e.g. read by readinto()).

    @serhiy-storchaka serhiy-storchaka added extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Dec 6, 2014
    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Dec 6, 2014

    In the event of calls back into Python code in multithreaded execution (or GIL release), this would mean you no longer have guarantees as to the contents (or even the validity) of the pointer you get back. I'd think the only safe way to accept mutable buffers would be to use the s*, z*, y* codes, which lock the buffer to prevent resize/destruction. Do we want to open segfault vulnerabilities in arbitrary functions?

    @serhiy-storchaka
    Copy link
    Member Author

    That is was the patch does. Convert from s# to s* etc.

    See also bpo-22896 about potential bugs with the use of pointers to unlocked
    buffers.

    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Dec 6, 2014

    Ah, sorry. Should have examined patch. I thought you were making a change to the behavior of s#, z#, y and y#, not converting actual uses of them. Again, sorry.

    @pitrou
    Copy link
    Member

    pitrou commented Dec 7, 2014

    +1 on the principle. I haven't looked at the patch.

    @vadmium
    Copy link
    Member

    vadmium commented Jan 5, 2015

    If this goes ahead, it would be nice adding notes to the documentation saying that bytearray() or whatever was previously not supported. There are APIs in Python 2.6 that had similar treatment with no documentation updates, and I keep being bitten by it.

    @serhiy-storchaka
    Copy link
    Member Author

    Added versionchanged directives and fixed other documentation.

    @vadmium
    Copy link
    Member

    vadmium commented Mar 20, 2015

    I had a closer look at the changes, and most of them seem good. However I think I found one leak in fcntl(); see Reitveld.

    I noticed there is no test for “ossaudiodev”. Would that be too hard, or is it just an oversight?

    @serhiy-storchaka
    Copy link
    Member Author

    Thank you for your review Martin. Yes, the problem with fcntl() is not so easy and I'll left it for the separate issue.

    I noticed there is no test for “ossaudiodev”. Would that be too hard, or is it just an oversight?

    It is hard because test_ossaudiodev is not designed to just apply the test with different type of data, and this test doesn't work on my computer at all (no /dev/dsp).

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Mar 20, 2015

    New changeset 17c77938c4e2 by Serhiy Storchaka in branch 'default':
    Issue bpo-23001: Few functions in modules mmap, ossaudiodev, socket, ssl, and
    https://hg.python.org/cpython/rev/17c77938c4e2

    @serhiy-storchaka serhiy-storchaka self-assigned this Mar 20, 2015
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Mar 20, 2015

    New changeset 11548e1ac920 by Victor Stinner in branch 'default':
    Issue bpo-23001: Fix typo
    https://hg.python.org/cpython/rev/11548e1ac920

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Mar 20, 2015

    New changeset d478a2a5738a by Victor Stinner in branch 'default':
    Issue bpo-23709, bpo-23001: ossaudiodev now uses Py_ssize_t for sizes instead of int
    https://hg.python.org/cpython/rev/d478a2a5738a

    @serhiy-storchaka
    Copy link
    Member Author

    Thanks Victor.

    @vstinner
    Copy link
    Member

    Thanks Victor.

    You're welcome, I just worked recently on ossaudiodev for _Py_read/write functions functions. But I'm not able to run ossaudiodev neither :-( I hope that the test runs on some buildbots (FreeBSD?).

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    extension-modules C modules in the Modules dir type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @vstinner @vadmium @pitrou @serhiy-storchaka