This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: popen4 doesn't close filedescriptors when in Threads
Type: Stage:
Components: Library (Lib) Versions: Python 2.2
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: facundobatista, gaul, mdoudoroff, nnorwitz
Priority: normal Keywords:

Created on 2003-07-09 18:36 by mdoudoroff, last changed 2022-04-10 16:09 by admin. This issue is now closed.

Messages (5)
msg17009 - (view) Author: martin doudoroff (mdoudoroff) Date: 2003-07-09 18:36
Running the following code under Linux will result in a
crash on the 508th thread started. The error is

    OSError: [Errno 24] Too many open files

The nature of the bug seems to be that Python isn't
closing filedescriptors cleanly when running a thread.

---------------------------------------
import os
from threading import Thread

class Crash(Thread):
    def run(self):
        a = os.popen4('ls')
        b = a[1].read()

        # uncommenting these lines fixes the problem
        #     but this isn't really documented as far as
        #     we can tell...
        # a[0].close()
        # a[1].close()

for i in range(1000):
    t = Crash()
    t.start()

    while t.isAlive():
         pass

    print i
---------------------------------------

The same code without threads (Crash as a plain class)
doesn't crash, so the descriptor must be getting taken
care of when the run() method is exited.

import os

class Crash:
    def run(self):
        a = os.popen4('ls')
        b = a[1].read()

for i in range(1000):
    t = Crash()
    t.run()

    print i
msg17010 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2003-07-10 03:20
Logged In: YES 
user_id=33168

I can't duplicate this on Redhat 9.  What OS, what version
of glibc and what kernel are you using?  Does it always
crash on the 508th iteration?

I tested with both 2.2.3 and 2.3b2 from CVS without
problems.  I even used ulimit to set my open files to 10.

Can you try the patch in bug #761888 to see if that helps? 
http://pythong.org/sf/761888 
msg17011 - (view) Author: Andrew Gaul (gaul) Date: 2003-10-01 18:51
Logged In: YES 
user_id=139865

Duplicated with Python 2.3 on Red Hat 7.3 using
glibc-2.2.5-43.  Popen3.{poll,wait} are written under the
incorrect assumption that waitpid can monitor any process in
the same process group, when it only works for immediate
children.  _active.remove is never called, so Popen3 objects
are never destroyed and the associated file descriptors are
not returned to the operating system.

A general solution for Popen[34] is not obvious to me.  With
patch #816059, popen2.popen[234] plugs the _active leak,
which in turn returns the file descriptors to the operating
system when the file objects that popen2.popen[234] return
go out of scope.
msg17012 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2005-01-15 20:48
Logged In: YES 
user_id=752496

Please, could you verify if this problem persists in Python 2.3.4
or 2.4?

If yes, in which version? Can you provide a test case?

If the problem is solved, from which version?

Note that if you fail to answer in one month, I'll close this bug
as "Won't fix".

Thank you! 

.    Facundo
msg17013 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2005-01-15 20:48
Logged In: YES 
user_id=752496

Works fine to me:

Python 2.3.4 (#1, Oct 26 2004, 16:42:40)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2

with glibc-2.3.4-2
History
Date User Action Args
2022-04-10 16:09:54adminsetgithub: 38829
2003-07-09 18:36:58mdoudoroffcreate