Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't read data from Transport after asyncio.SubprocessStreamProtocol closes #73890

Closed
SethMichaelLarson mannequin opened this issue Mar 2, 2017 · 4 comments
Closed

Can't read data from Transport after asyncio.SubprocessStreamProtocol closes #73890

SethMichaelLarson mannequin opened this issue Mar 2, 2017 · 4 comments
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@SethMichaelLarson
Copy link
Mannequin

SethMichaelLarson mannequin commented Mar 2, 2017

BPO 29704
Nosy @1st1, @SethMichaelLarson
PRs
  • bpo-29704: Fix asyncio.SubprocessStreamProtocol closing #405
  • bpo-29704: Fix asyncio.SubprocessStreamProtocol closing  #415
  • bpo-29704: Fix asyncio.SubprocessStreamProtocol closing #416
  • [Do Not Merge] Sample of CPython life with blurb. #703
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-03-03.23:12:44.377>
created_at = <Date 2017-03-02.22:24:23.132>
labels = ['type-bug', 'expert-asyncio']
title = "Can't read data from Transport after asyncio.SubprocessStreamProtocol closes"
updated_at = <Date 2017-03-31.16:36:23.221>
user = 'https://github.com/SethMichaelLarson'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:23.221>
actor = 'dstufft'
assignee = 'none'
closed = True
closed_date = <Date 2017-03-03.23:12:44.377>
closer = 'yselivanov'
components = ['asyncio']
creation = <Date 2017-03-02.22:24:23.132>
creator = 'SethMichaelLarson'
dependencies = []
files = []
hgrepos = []
issue_num = 29704
keywords = []
message_count = 4.0
messages = ['288839', '290327', '290333', '290335']
nosy_count = 2.0
nosy_names = ['yselivanov', 'SethMichaelLarson']
pr_nums = ['405', '415', '416', '703', '552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29704'
versions = []
    @SethMichaelLarson
    Copy link
    Mannequin Author

    SethMichaelLarson mannequin commented Mar 2, 2017

    Copied from python/asyncio#484

    """
    From https://bugs.python.org/issue23242#msg284930

    The following script is used to reproduce the bug:

    import asyncio

async def execute():
    process = await asyncio.create_subprocess_exec(
        "timeout", "0.1", "cat", "/dev/urandom", stdout=asyncio.subprocess.PIPE)

    while True:
        data = await process.stdout.read(65536)
        print('read %d bytes' % len(data))
        if data:
            await asyncio.sleep(0.3)
        else:
            break

    asyncio.get_event_loop().run_until_complete(execute())

    will produce following output and terminate with exception:

    read 65536 bytes
read 65536 bytes
Traceback (most recent call last):
  File "read_subprocess.py", line 18, in <module>
    asyncio.get_event_loop().run_until_complete(execute())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "read_subprocess.py", line 9, in execute
    data = await process.stdout.read(65536)
  File "/usr/lib/python3.6/asyncio/streams.py", line 634, in read
    self._maybe_resume_transport()
  File "/usr/lib/python3.6/asyncio/streams.py", line 402, in _maybe_resume_transport
    self._transport.resume_reading()
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 401, in resume_reading
    self._loop._add_reader(self._fileno, self._read_ready)
AttributeError: 'NoneType' object has no attribute '_add_reader'

    When the process exits https://github.com/python/asyncio/blob/master/asyncio/unix_events.py#L444 is called which sets this._loop = None
    Next time read() is called on the pipe the above exception is thrown.
    I have tried to fix this issue myself but would sometimes have read terminate too early and miss the last chunks of data.
    """

    • BotoX

    @SethMichaelLarson SethMichaelLarson mannequin added topic-asyncio type-bug An unexpected behavior, bug, or error labels Mar 2, 2017
    @1st1 1st1 closed this as completed Mar 3, 2017
    @1st1
    Copy link
    Member

    1st1 commented Mar 24, 2017

    New changeset f7f024a by Yury Selivanov (Seth M. Larson) in branch '3.5':
    bpo-29704: Fix asyncio.SubprocessStreamProtocol closing (#405)
    f7f024a

    @1st1
    Copy link
    Member

    1st1 commented Mar 24, 2017

    New changeset 604faba by Yury Selivanov (Seth M. Larson) in branch '3.6':
    bpo-29704: Fix asyncio.SubprocessStreamProtocol closing (#405)
    604faba

    @1st1
    Copy link
    Member

    1st1 commented Mar 24, 2017

    New changeset 481cb70 by Yury Selivanov (Seth M. Larson) in branch 'master':
    bpo-29704: Fix asyncio.SubprocessStreamProtocol closing (#405)
    481cb70

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    topic-asyncio type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @1st1