This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: 3.5.0b3 Windows accept() on unready non-blocking socket raises PermissionError [now need unit test]
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: bryangeneolson, eryksun, larry, neologix, paul.moore, pitrou, python-dev, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Priority: high Keywords:

Created on 2015-07-27 02:13 by bryangeneolson, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (10)
msg247452 - (view) Author: Bryan G. Olson (bryangeneolson) * Date: 2015-07-27 02:13
In Python 3.4 on Windows 7, the code:

    import socket
    sock = socket.socket()
    sock.bind(('127.0.0.1', 52384))
    sock.listen(5)
    sock.setblocking(False)
    csock, addr = sock.accept()

Raised:

    Traceback (most recent call last):
      File "socket_bug_test.py", line 8, in <module>
        csock, addr = sock.accept()
      File "c:\bin\Python34\lib\socket.py", line 187, in accept
        fd, addr = self._accept()
    BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately

In Python 3.5.0b3 on the same system, it is raising:

    Traceback (most recent call last):
      File "socket_bug_test.py", line 8, in <module>
        csock, addr = sock.accept()
      File "c:\bin\Python35\lib\socket.py", line 195, in accept
        fd, addr = self._accept()
    PermissionError: [WinError 5] Access is denied

This is a problem for other parts of the Standard Library. I hit it playing with asyncio, where in Lib\asyncio\selector_events.py the function BaseSelectorEventLoop._sock_accept() is prepared for an unready socket to raise BlockingIOError or InterruptedError, but does not catch the WinError:

        try:
            conn, address = sock.accept()
            conn.setblocking(False)
        except (BlockingIOError, InterruptedError):
            self.add_reader(fd, self._sock_accept, fut, True, sock)
        except Exception as exc:
            fut.set_exception(exc)
        else:
            fut.set_result((conn, address))

There are other calls to accept in accept() in asyncio, that I haven't tested but also look adversely affected.

The older asyncore module looks to have a similar problem. In dispatcher.accept(): 

    def accept(self):
        # XXX can return either an address pair or None
        try:
            conn, addr = self.socket.accept()
        except TypeError:
            return None
        except OSError as why:
            if why.args[0] in (EWOULDBLOCK, ECONNABORTED, EAGAIN):
                return None
            else:
                raise
        else:
            return conn, addr

The 'except OSError as why' will catch the WinError, but why.args[0] will be errno.EACCES = 13, permission denied, which is not equal to any of anticipated errors.


My system according to Python:

    >>> import sys
    >>> sys.version
    '3.5.0b3 (default, Jul  5 2015, 07:00:46) [MSC v.1900 64 bit (AMD64)]'
    >>> sys.getwindowsversion()
    sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1')

That's Windows 7 Professional, 64-bit, with Service pack 1. It has an AND Phenom II X6 1055T Processor 2.8 GHz, and 8GB of memory.
msg247453 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2015-07-27 02:50
Thanks for the report!  I'm pretty sure I've noticed the effects of this, but hadn't had a chance to look into it.  With your nice small test case, I can start bisecting and try to figure out what caused it.
msg247475 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2015-07-27 17:00
According to hg bisect:

The first bad revision is:
changeset:   95361:358a2bcd0d0b
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Wed Apr 01 21:57:09 2015 +0200
summary:     Issue #23834: Add sock_call() helper function

Nosy list from that issue added here.

I don't understand that patch well enough to suggest any kind of fix, but I'm happy to run tests for anyone who can fix it.

I think this is a serious enough issue to block the next pre-release of 3.5.
msg247479 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-07-27 19:47
The permission error comes from calling SetHandleInformation on an invalid socket value. The code shouldn't make it that far. The problem starts earlier in sock_accept_impl. This function checks whether SOCKET_T ctx->result is non-negative to determine whether the accept call succeeded, but SOCKET_T is unsigned on Windows.
msg247489 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-07-27 21:39
New changeset cd60eccaa331 by Victor Stinner in branch '3.5':
Issue #24732, #23834: Fix sock_accept_impl() on Windows
https://hg.python.org/cpython/rev/cd60eccaa331
msg247491 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-07-27 21:46
> According to hg bisect: (...)
> summary:     Issue #23834: Add sock_call() helper function

Oh, I'm not too surprised. I had to modify _deeply_ socketmodule.c to implement the PEP 475. I stressed the code on Linux, but on Windows I only ran Python test suite and asyncio test suite.

The surprising part is that the bug was not detected by any test :-(

We need at least one more unit test for it.

> The problem starts earlier in sock_accept_impl. This function checks whether SOCKET_T ctx->result is non-negative to determine whether the accept call succeeded, but SOCKET_T is unsigned on Windows.

Thanks, good analysis. I checked accept() documentation and Python 3.4 source code: yeah, we must check for INVALID_SOCKET. It's a dummy copy/paste failure. I copied the code from another sock_xxx_impl() function, and I didn't notice that the error condition is different for accept(). The code fails in some cases on Windows, but not always. Maybe only on 64-bit?

> My system according to Python:
> ...
> '3.5.0b3 (default, Jul  5 2015, 07:00:46) [MSC v.1900 64 bit (AMD64)]'

My Windows is also 64-bit but my Visual Studio version (2010 Express) was limited to 32-bit :-(

Right now, I cannot develop on Windows: Visual Studio 2010 doesn't want to compile Python anymore (issue #24737). I'm downloading Windows 8.1 and I will try Visual Studio 2015. So I pushed a fix without being able to test it :-(
msg247494 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-07-27 23:03
LGTM on Windows 7:

    Python 3.5.0b4+ (default, Jul 27 2015, 17:46:34) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import socket
    >>> sock = socket.socket()
    >>> sock.bind(('127.0.0.1', 52380))
    >>> sock.listen(5)
    >>> sock.setblocking(False)
    >>> csock, addr = sock.accept()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Source\cpython\3.5\lib\socket.py", line 195, in accept
        fd, addr = self._accept()
    BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately
msg247495 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-07-28 00:30
> LGTM on Windows 7:

I just compiled Python default (3.6) on Windows 8.1 with Visual Studio 2015: I confirm, the example of the original message now works as expected (raise "BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately").

It still lacks an unit test.
msg248303 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-08 21:55
If this is now fixed but still needs a unit test, can I change it from "release blocker" to "high" priority?
msg341888 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2019-05-08 15:23
3.4 is now EOL, so the 3.4regression tag goes away too.
History
Date User Action Args
2022-04-11 14:58:19adminsetgithub: 68920
2019-05-27 23:56:21vstinnersetstatus: open -> closed
resolution: fixed
stage: resolved
2019-05-08 15:23:44larrysetkeywords: - 3.4regression

messages: + msg341888
2015-08-19 05:31:22vstinnersettitle: 3.5.0b3 Windows accept() on unready non-blocking socket raises PermissionError -> 3.5.0b3 Windows accept() on unready non-blocking socket raises PermissionError [now need unit test]
2015-08-09 10:11:30larrysetpriority: release blocker -> high
2015-08-08 21:55:24larrysetnosy: + larry
messages: + msg248303
2015-07-28 00:30:07vstinnersetmessages: + msg247495
2015-07-27 23:03:08eryksunsetmessages: + msg247494
2015-07-27 21:46:22vstinnersetmessages: + msg247491
2015-07-27 21:39:32python-devsetnosy: + python-dev
messages: + msg247489
2015-07-27 19:47:55eryksunsetnosy: + eryksun
messages: + msg247479
2015-07-27 17:00:32zach.waresetpriority: high -> release blocker

nosy: + pitrou, vstinner, neologix, serhiy.storchaka
messages: + msg247475

keywords: + 3.4regression
2015-07-27 02:50:54zach.waresetpriority: normal -> high

messages: + msg247453
versions: + Python 3.6
2015-07-27 02:13:48bryangeneolsoncreate