Author gregory.p.smith
Recipients gregory.p.smith, larry, sstewartgallus
Date 2014-05-31.22:23:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1401574990.13.0.128202577324.issue21618@psf.upfronthosting.co.za>
In-reply-to
Content
There appear to be a two bugs here, depending on which platform subprocess is being used on.

1) on systems where it uses /prod/self/fd, /dev/fd or similar:

 It should not pay attention to end_fd at all.  It knows the list of actual open fds and should use that.  If possible, consider detecting and avoiding closing valgrind fds; but that is a special case for a valgrind bug and likely not worth it.

2) on systems without a way to get the list of open file descriptors:

 The sysconf("SC_OPEN_MAX") value is only saved at module import time but may be changed up or down at runtime by the process by using the setrlimit(RLIMIT_NOFILE, ...) libc call.  what sysconf returns is the same as the current rlim_cur setting.  (at least on Linux where this code path wouldn't actually be taken because #1 is available).

 possible solution: call getrlimit(RLIMIT_NOFILE) and use rlim_max instead of sysconf("SC_OPEN_MAX") at module import time.
 caveat: rlim_max can be raised by processes granted that capbility.  It is impossible to do anything about that in this scenario given we're operating w/o a way to get a list of open fds.
 impact: only on OSes that lack implementations that get a list of open fds as in #1 above. so... nothing that anyone really uses unless they choose to come contribute support for that themselves. (linux, bsd and os x all fall into #1 above)

Neither of these are likely scenarios so I wouldn't consider this a high priority to fix but it should be done.  Most code never ever touches its os resource limits.  getrlimit and setrlimit are not exposed in the os module; you must use ctypes or an extension module to call them from Python:

import ctypes
class StructRLimit(ctypes.Structure):
  _fields_ = [('rlim_cur', ctypes.c_ulong), ('rlim_max', ctypes.c_ulong)]
libc = ctypes.cdll.LoadLibrary('libc.so.6')
RLIMIT_NOFILE = 7  # Linux
limits = StructRLimit()
assert libc.getrlimit(RLIMIT_NOFILE, ctypes.byref(limits)) == 0
print(limits.rlim_cur, limits.rlim_max)
limits.rlim_cur = limits.rlim_max
assert libc.setrlimit(RLIMIT_NOFILE, ctypes.byref(limits)) == 0
History
Date User Action Args
2014-05-31 22:23:10gregory.p.smithsetrecipients: + gregory.p.smith, larry, sstewartgallus
2014-05-31 22:23:10gregory.p.smithsetmessageid: <1401574990.13.0.128202577324.issue21618@psf.upfronthosting.co.za>
2014-05-31 22:23:09gregory.p.smithlinkissue21618 messages
2014-05-31 22:23:06gregory.p.smithcreate