This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "Too many open files" on macOS buildbot
Type: Stage: resolved
Components: Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mattbillenstein, pablogsal, pitrou, zach.ware
Priority: normal Keywords:

Created on 2021-03-20 21:51 by pitrou, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg389184 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-03-20 21:51
See https://buildbot.python.org/all/#/builders/366/builds/960/steps/5/logs/stdio
msg389208 - (view) Author: Matt Billenstein (mattbillenstein) Date: 2021-03-21 02:46
I don't see that error message - where are you seeing it?
msg389209 - (view) Author: Matt Billenstein (mattbillenstein) Date: 2021-03-21 03:10
In any case - I've raised file/process limits on the buildbot.
msg389211 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-03-21 03:27
>> I don't see that error message - where are you seeing it?


You need to search for it in the full log:


stderr:
---
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/spawn.py", line 176, in get_preparation_data
    dir=os.getcwd(),
OSError: [Errno 24] Too many open files
msg389212 - (view) Author: Matt Billenstein (mattbillenstein) Date: 2021-03-21 03:35
Ah, word, was looking at stdout -- 256 is pretty low, raised it to 200k...
msg389224 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-03-21 11:39
Thank you!
History
Date User Action Args
2022-04-11 14:59:43adminsetgithub: 87738
2021-03-21 11:39:20pitrousetstatus: open -> closed
resolution: fixed
stage: resolved
2021-03-21 11:39:13pitrousetmessages: + msg389224
2021-03-21 03:35:16mattbillensteinsetmessages: + msg389212
2021-03-21 03:27:36pablogsalsetmessages: + msg389211
2021-03-21 03:10:02mattbillensteinsetmessages: + msg389209
2021-03-21 02:46:37mattbillensteinsetmessages: + msg389208
2021-03-20 21:51:02pitroucreate