Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import socket
   ...: s1, s2 = socket.socketpair()
   ...: import asyncio
   ...: async def test():
   ...:     r1, w1 = await asyncio.open_connection(sock=s1)
   ...:     r2, w2 = await asyncio.open_connection(sock=s2)
   ...:     s1.close()
   ...: asyncio.run(test())
Exception in callback _ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None))
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None))>
Traceback (most recent call last):
  File "c:\python39\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "c:\python39\lib\asyncio\proactor_events.py", line 162, in _call_connection_lost
    self._sock.shutdown(socket.SHUT_RDWR)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

I cannot reproduce on a Debian machine. Seems to be a Windows component issue?

It reproduced on a windows machine

Please consider passing 'sock' argument as the ownership transfer.

You should not perform any action on 'sock' object directly anymore.
This is true for all asyncio API.

We are hitting the same traceback with mitmproxy on Windows without ever passing a socket object to open_connection. In other words, this can be triggered without performing any action on 'sock' due to some race conditions in the Windows network stack. I unfortunately don't have a better repro for that race other than what Daniel provided.

asvetlov: Sorry if I articulated myself badly, but I do think this is a valid bug. It's unfortunately hard to provide a better repro (I tried), but we are hitting this regularly when mitmproxy is accepting connections under heavy load. We're just calling asyncio.start_server(handler, "127.0.0.1", 8080) in mitmproxy and never interact with the underlying socket object.

Here are some observations that are true for all crashes:

• The socket fileno is -1 when it crashes.
• _call_connection_lost is called by _ProactorBasePipeTransport.close, which is called by _ProactorBasePipeTransport.__del__ [1]
• There are no previous calls to _call_connection_lost.
• Windows only, loopback connections in our case.
• Wireshark shows that client and server are first happily exchanging packets. At some point the client sends a FIN, which the Python server ACKs immediately. A few seconds later the Python server sends a FIN back.

An obvious fix without understanding the root cause would be to check fileno in

[1]

Maximilian, thanks for the investigation.

A check for 'fileno != -1' seems correct to me.
Would you prepare a pull request?

New changeset 7015541 by Maximilian Hils in branch 'main':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
7015541

New changeset 88c243f by Miss Islington (bot) in branch '3.10':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
88c243f

New changeset 64a68c3 by Miss Islington (bot) in branch '3.9':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
64a68c3

Thanks!

Thank you for the super quick turnaround.
My heart goes out to you and everyone else in Ukraine.