Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.8 improperly warns about closing properly closed streams #82710

Closed
RonFrederick mannequin opened this issue Oct 19, 2019 · 6 comments
Closed

Python 3.8 improperly warns about closing properly closed streams #82710

RonFrederick mannequin opened this issue Oct 19, 2019 · 6 comments
Labels
3.8 only security fixes 3.9 only security fixes release-blocker topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@RonFrederick
Copy link
Mannequin

RonFrederick mannequin commented Oct 19, 2019

BPO 38529
Nosy @asvetlov, @cjrh, @ambv, @1st1, @thehesiod, @miss-islington
PRs
  • bpo-38529: Fix asyncio stream warning #17474
  • [3.8] bpo-38529: Fix asyncio stream warning (GH-17474) #17492
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-12-07.11:40:47.255>
    created_at = <Date 2019-10-19.18:40:28.968>
    labels = ['type-bug', '3.8', '3.9', 'release-blocker', 'expert-asyncio']
    title = 'Python 3.8 improperly warns about closing properly closed streams'
    updated_at = <Date 2019-12-07.11:40:47.254>
    user = 'https://bugs.python.org/RonFrederick'

    bugs.python.org fields:

    activity = <Date 2019-12-07.11:40:47.254>
    actor = 'asvetlov'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-12-07.11:40:47.255>
    closer = 'asvetlov'
    components = ['asyncio']
    creation = <Date 2019-10-19.18:40:28.968>
    creator = 'Ron Frederick'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38529
    keywords = ['patch']
    message_count = 6.0
    messages = ['354953', '355481', '355482', '355485', '357968', '357970']
    nosy_count = 8.0
    nosy_names = ['falsetru', 'asvetlov', 'cjrh', 'lukasz.langa', 'yselivanov', 'thehesiod', 'Ron Frederick', 'miss-islington']
    pr_nums = ['17474', '17492']
    priority = 'release blocker'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue38529'
    versions = ['Python 3.8', 'Python 3.9']

    @RonFrederick
    Copy link
    Mannequin Author

    RonFrederick mannequin commented Oct 19, 2019

    In testing AsyncSSH against Python 3.8, I noticed a large number of the following errors, even though I was properly closing streams before the objects holding them were garbage-collected.

    An open stream object is being garbage collected; call "stream.close()" explicitly.
    

    After some investigation, the problem appears to be that closing a stream is not good enough to prevent the error. The check in asyncio doesn't properly handle the case where the stream is closing, but has not fully closed. Here's a simple test program that demonstrates this:

    import asyncio
    
    async def tcp_client():
        reader, writer = await asyncio.open_connection('127.0.0.1', 22)
    
        writer.close()
    
    asyncio.run(tcp_client())

    It's possible to avoid this message by awaiting on writer.wait_closed(), but wait_closed() doesn't exist until Python 3.7, making it very difficult to write portable code and still avoid this message.

    @RonFrederick RonFrederick mannequin added 3.8 only security fixes topic-asyncio type-bug An unexpected behavior, bug, or error labels Oct 19, 2019
    @RonFrederick
    Copy link
    Mannequin Author

    RonFrederick mannequin commented Oct 27, 2019

    I think the following change might address this problem:


    *** 233,239 ****

      def \_on_reader_gc(self, wr):
          transport = self.\_transport
    

    ! if transport is not None:
    # connection_made was called
    context = {
    'message': ('An open stream object is being garbage '
    --- 233,239 ----

      
          def _on_reader_gc(self, wr):
              transport = self._transport
    !         if transport is not None and not transport.is_closing():
                  # connection_made was called
                  context = {
                      'message': ('An open stream object is being garbage '

    @RonFrederick
    Copy link
    Mannequin Author

    RonFrederick mannequin commented Oct 27, 2019

    Sorry, I should have said that the change below was in the file asyncio/streams.py.

    @1st1
    Copy link
    Member

    1st1 commented Oct 27, 2019

    Yes, I've experienced this bug. We need to fix this in 3.8.1.

    @asvetlov asvetlov added the 3.9 only security fixes label Dec 5, 2019
    @asvetlov
    Copy link
    Contributor

    asvetlov commented Dec 7, 2019

    New changeset 7ddcd0c by Andrew Svetlov in branch 'master':
    bpo-38529: Fix asyncio stream warning (GH-17474)
    7ddcd0c

    @miss-islington
    Copy link
    Contributor

    New changeset 7fde4f4 by Miss Islington (bot) in branch '3.8':
    bpo-38529: Fix asyncio stream warning (GH-17474)
    7fde4f4

    @asvetlov asvetlov closed this as completed Dec 7, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes release-blocker topic-asyncio type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants