This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_selectors.PollSelectorTestCase failing on macOS 10.13.3
Type: Stage: resolved
Components: Tests Versions: Python 3.6
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: n8henrie, ned.deily
Priority: normal Keywords:

Created on 2018-03-09 16:53 by n8henrie, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (9)
msg313487 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-09 16:53
Failing for me on latest 3.6, 3.6.1, 3.5.5, may be related to https://bugs.python.org/issue32517, presumably a change on macOS KQueue stuff.

Can anyone else on macOS 10.13.3 see if they can reproduce?

```
make clean && ./configure --with-pydebug && make -j
./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase
```
msg313488 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-09 16:55
Traceback:

```
  File "cpython/Lib/test/test_selectors.py", line 453, in test_above_fd_setsize
    self.assertEqual(NUM_FDS // 2, len(s.select()))
  File "cpython/Lib/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
OSError: [Errno 22] Invalid argument
```
msg313489 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-09 17:26
Sorry, I am not seeing this failure on 10.13.3 with either current 3.6 head or with v3.6.1.  Didn't try anything else.
msg313840 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-14 20:50
Hmmm, still failing for me. I wonder if it's something specific to my machine.

```
git reset --hard 3.6 && make clean && git pull && ./configure --with-pydebug && make -j && ./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase.test_above_fd_setsize
```

Related:

- https://bugs.python.org/issue18963
- https://bugs.python.org/issue21901
msg313844 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-14 23:20
I'm not sure what to suggest other than taking a close look at and instrumenting that part of test_selectors.  One thing you could try is seeing what the call to resource.getrlimit(resource.RLIMIT_NOFILE) is returning.  You can also look at the values for your system's kernel by using the sysctl utility; see man sysctl.  For example, on my system:

>>> resource.getrlimit(resource.RLIMIT_NOFILE)
(7168, 9223372036854775807)

$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 36864

That may or may not be relevant.

Long shot: are you running under an account that does not have administrator privs?
msg313856 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-15 08:09
Thanks for the response -- I'll keep looking, feel free to close since it's
not being reproduced.

```
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 64000
$ ./python.exe -c 'import resource;
print(resource.getrlimit(resource.RLIMIT_NOFILE))'
(64000, 524288)
```

Nope, admin access.
msg313951 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-16 15:08
Update -- I found the following plist at `/Library/LaunchDaemons/limit.maxfiles.plist`:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
      <string>launchctl</string>
      <string>limit</string>
      <string>maxfiles</string>
      <string>64000</string>
      <string>524288</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
  </dict>
</plist>
```

I think I made this file at some point to deal with an error about insufficient file descriptors while playing with asyncio / sockets. After `launchctl unload`ing it and rebooting **the test now passes**, and I now see a *much* higher hard limit (similar to that posted by Ned) with the `getrlimit()` command, and `unlimited` as the hard limit with `launchtl limit`.

I can reproduce the failure by restoring my prior settings:

```bash
$ sudo launchctl limit maxfiles 64000 524288
$ ulimit -n 64000
$ 
$ ./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase.test_above_fd_setsize
test_above_fd_setsize (test.test_selectors.PollSelectorTestCase) ... ERROR

======================================================================
ERROR: test_above_fd_setsize (test.test_selectors.PollSelectorTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/n8henrie/git/cpython/Lib/test/support/__init__.py", line 600, in wrapper
    return func(*args, **kw)
  File "/Users/n8henrie/git/cpython/Lib/test/test_selectors.py", line 453, in test_above_fd_setsize
    self.assertEqual(NUM_FDS // 2, len(s.select()))
  File "/Users/n8henrie/git/cpython/Lib/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
OSError: [Errno 22] Invalid argument

----------------------------------------------------------------------
Ran 1 test in 9.771s

FAILED (errors=1)

```
msg313952 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-03-16 15:49
Glad you found the problem - and that it wasn't a Python bug!
msg314149 - (view) Author: Nathan Henrie (n8henrie) * Date: 2018-03-20 16:06
I've continued looking into this.

If you have your limits configured higher than default, as I did (and which seems to be working fine):

```
sudo launchctl limit maxfiles 64000 524288
ulimit -Sn 64000
```

then you'll find that having a soft limit (`NUM_FDS`, prior to the `-=32`) greater than 10273 causes the error I was seeing.

First, I found this number by passing in NUM_FDS as an envvar to the unittest and looping in a bash script, but I have since extracted some of the relevant portions to a separate python script:

```python
import selectors
import socket
import sys

def main(NUM_FDS):
    s = selectors.PollSelector()

    NUM_FDS -= 32

    sockets = []

    try:
        for i in range(NUM_FDS // 2):
            try:
                rd, wr = socket.socketpair()
                sockets.extend([rd, wr])
            except OSError as e:
                print(f"err 2: {type(e)} {e}")
                # too many FDs, skip - note that we should only catch EMFILE
                # here, but apparently *BSD and Solaris can fail upon connect()
                # or bind() with EADDRNOTAVAIL, so let's be safe
                print("FD limit reached")

            try:
                s.register(rd, selectors.EVENT_READ)
                s.register(wr, selectors.EVENT_WRITE)
            except OSError as e:
                print(f"err 3: {type(e)} {e}")
                if e.errno == errno.ENOSPC:
                    # this can be raised by epoll if we go over
                    # fs.epoll.max_user_watches sysctl
                    print("FD limit reached")
                raise

        try:
            slen = len(s.select())
        except OSError as e:
            print(f"err 4: {type(e)} {e}")
            raise
        print(f"asserting {NUM_FDS // 2} == {slen}")
        assert NUM_FDS // 2 == slen

    finally:
        for sock in sockets:
            sock.close()
        s.close()
        print("Closed")

if __name__ == "__main__":
    main(int(sys.argv[1]))
```


```shellsession
$ ./python.exe tester.py 10273
asserting 5120 == 5120
Closed
$ ./python.exe tester.py 10274
err 4: <class 'OSError'> [Errno 22] Invalid argument
Closed
Traceback (most recent call last):
  File "tester.py", line 50, in <module>
    main(int(sys.argv[1]))
  File "tester.py", line 36, in main
    slen = len(s.select())
  File "cpython/Lib/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
OSError: [Errno 22] Invalid argument
```

Tested and confirmed that I can provoke the error with the above launchctl / ulimit settings on my wife's Macbook Air.

I wonder if the 10273 limit I'm running up against has something to do with the 10240 number I keep running into as a default kern.maxfilesperproc (https://unix.stackexchange.com/a/350824/77904), given the `NUM_FDS -= 32` in the test.
History
Date User Action Args
2022-04-11 14:58:58adminsetgithub: 77217
2018-03-20 16:06:24n8henriesetmessages: + msg314149
2018-03-16 15:49:22ned.deilysetstatus: open -> closed
resolution: works for me
messages: + msg313952
2018-03-16 15:08:17n8henriesetstatus: closed -> open
resolution: works for me -> (no value)
messages: + msg313951
2018-03-16 02:15:19ned.deilysetstatus: open -> closed
resolution: works for me
stage: resolved
2018-03-15 08:09:30n8henriesetmessages: + msg313856
2018-03-14 23:20:58ned.deilysetmessages: + msg313844
2018-03-14 20:50:54n8henriesetmessages: + msg313840
2018-03-09 17:26:11ned.deilysetnosy: + ned.deily
messages: + msg313489
2018-03-09 16:55:24n8henriesetmessages: + msg313488
2018-03-09 16:53:58n8henriecreate