classification
Title: bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system
Type: Stage: resolved
Components: macOS Versions: Python 3.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: ned.deily, ronaldoussoren, vstinner
Priority: normal Keywords:

Created on 2017-12-12 15:40 by vstinner, last changed 2018-05-28 08:48 by ned.deily. This issue is now closed.

Messages (6)
msg308141 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-12 15:40
http://buildbot.python.org/all/#/builders/49/builds/67

...
http://buildbot.python.org/all/#/builders/49/builds/67
...
Installing collected packages: sphinxcontrib-websupport, setuptools, snowballstemmer, docutils, imagesize, urllib3, chardet, certifi, idna, requests, pytz, babel, alabaster, MarkupSafe, Jinja2, six, Pygments, Sphinx, blurb
  Found existing installation: setuptools 28.8.0
    Uninstalling setuptools-28.8.0:
      Successfully uninstalled setuptools-28.8.0
  Running setup.py install for MarkupSafe: started
    Running setup.py install for MarkupSafe: finished with status 'done'
Successfully installed Jinja2-2.10 MarkupSafe-1.0 Pygments-2.2.0 Sphinx-1.6.5 alabaster-0.7.10 babel-2.5.1 blurb-1.0.5 certifi-2017.11.5 chardet-3.0.4 docutils-0.14 idna-2.6 imagesize-0.7.1 pytz-2017.3 requests-2.18.4 setuptools-38.2.4 six-1.11.0 snowballstemmer-1.2.1 sphinxcontrib-websupport-1.0.1 urllib3-1.22
The venv has been created in the ./venv directory

Exception occurred:
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/util/osutil.py", line 230, in getcwd
OSError: [Errno 23] Too many open files in system
Traceback (most recent call last):
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/cmdline.py", line 306, in main
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/application.py", line 339, in build
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 331, in build_update
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 344, in build
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 584, in update
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 603, in _read_serial
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 735, in read_doc
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/application.py", line 489, in emit
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/events.py", line 79, in emit
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/environment/collectors/dependencies.py", line 43, in process_doc
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/util/osutil.py", line 230, in getcwd
OSError: [Errno 23] Too many open files in system

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./venv/bin/sphinx-build", line 11, in <module>
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/__init__.py", line 71, in main
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/__init__.py", line 113, in build_main
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/cmdline.py", line 309, in main
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/cmdline.py", line 105, in handle_exception
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/util/__init__.py", line 219, in save_traceback
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 335, in mkstemp
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 130, in _sanitize_params
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 296, in gettempdir
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 231, in _get_default_tempdir
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '.']
make: *** [build] Error 1
Traceback (most recent call last):
mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -b html -d build/doctrees -D latex_elements.papersize=  . build/html 
Running Sphinx v1.6.5
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 465 source files that are out of date
updating environment: 465 added, 0 changed, 0 removed
reading sources... [  0%] about
reading sources... [  0%] bugs

...
    main()
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Mac/BuildScript/build-installer.py", line 1620, in main
    buildPythonDocs()
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Mac/BuildScript/build-installer.py", line 1093, in buildPythonDocs
    runCommand('make html PYTHON=venv/bin/python')
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Mac/BuildScript/build-installer.py", line 531, in runCommand
    raise RuntimeError("command failed: %s"%(commandline,))
RuntimeError: command failed: make html PYTHON=venv/bin/python
program finished with exit code 1
elapsedTime=1719.516712
msg308142 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-12 15:42
A different but maybe related failure on x86 Tiger 3.x:

http://buildbot.python.org/all/#/builders/30/builds/323

======================================================================
ERROR: test_rapid_restart (test.test_multiprocessing_fork.WithThreadsTestManagerRestart)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/_test_multiprocessing.py", line 2706, in test_rapid_restart
    manager.start()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/multiprocessing/managers.py", line 533, in start
    reader, writer = connection.Pipe(duplex=False)
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/multiprocessing/connection.py", line 517, in Pipe
    fd1, fd2 = os.pipe()
OSError: [Errno 23] Too many open files in system
======================================================================
ERROR: test_timeout (test.test_multiprocessing_fork.WithThreadsTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/_test_multiprocessing.py", line 1007, in test_timeout
    q = multiprocessing.Queue()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/multiprocessing/context.py", line 102, in Queue
    return Queue(maxsize, ctx=self.get_context())
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/multiprocessing/queues.py", line 41, in __init__
    self._reader, self._writer = connection.Pipe(duplex=False)
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/multiprocessing/connection.py", line 517, in Pipe
    fd1, fd2 = os.pipe()
OSError: [Errno 23] Too many open files in system
msg308407 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-15 15:38
Another different failure on "bolen-dmg-3.x". I don't understand the "RuntimeError: command failed: make" error :-(

http://buildbot.python.org/all/#/builders/69/builds/69

(...)
gcc-4.0 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -I/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/dmg/libraries/usr/local/include   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers  -IObjects -IInclude -IPython -I. -I/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Include -isysroot /Developer/SDKs/MacOSX10.4u.sdk    -DPy_BUILD_CORE -o Python/frozen.o /Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Python/frozen.c
rm -f libpython3.7m.a
ar rcs libpython3.7m.a Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o    Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/_operator.o  Modules/_collectionsmodule.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/signalmodule.o  Modules/_stat.o  Modules/timemodule.o  Modules/_threadmodule.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/_tracemalloc.o Modules/hashtable.o  Modules/symtablemodule.o  Modules/xxsubtype.o Python/frozen.o
/usr/bin/install -c -d -m 755 Python.framework/Versions/3.7
gcc-4.0 -o Python.framework/Versions/3.7/Python -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -L/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/dmg/libraries/usr/local/lib  -dynamiclib \
-all_load libpython3.7m.a -Wl,-single_module \
-install_name /Library/Frameworks/Python.framework/Versions/3.7/Python \
-compatibility_version 3.7 \
-current_version 3.7 \
-framework CoreFoundation -ldl  -framework CoreFoundation;
Traceback (most recent call last):
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Mac/BuildScript/build-installer.py", line 1659, in <module>
    main()
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Mac/BuildScript/build-installer.py", line 1612, in main
    buildPython()
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Mac/BuildScript/build-installer.py", line 1162, in buildPython
    runCommand(make_cmd)
  File "/Users/db3l/buildarea.dmg/bolen-dmg-3.x/build/Mac/BuildScript/build-installer.py", line 532, in runCommand
    raise RuntimeError("command failed: %s"%(commandline,))
RuntimeError: command failed: make
program finished with exit code 1
elapsedTime=1601.300462
msg317773 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-05-26 22:19
The macOS Tiger buildbot where all of these "too many files open" OS failures occurred has been retired.
msg317830 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-05-28 08:42
> The macOS Tiger buildbot where all of these "too many files open" OS failures occurred has been retired.

macOS Tiger is still supported, since we still have x86 Tiger buildbot, it's just the dmg buildbot on Tiger which has been retired. Right?
msg317832 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-05-28 08:48
Whether or not Tiger is supported, we no longer have any Tiger buildbots. We agreed to retire them earlier this year.
History
Date User Action Args
2018-05-28 08:48:37ned.deilysetmessages: + msg317832
2018-05-28 08:42:15vstinnersetmessages: + msg317830
2018-05-26 22:19:48ned.deilysetstatus: open -> closed
resolution: out of date
messages: + msg317773

stage: resolved
2017-12-15 15:38:35vstinnersetmessages: + msg308407
2017-12-12 15:42:03vstinnersetmessages: + msg308142
2017-12-12 15:40:45vstinnercreate