The test_subprocess fails since issue21618 on OpenBSD because the FD_DIR is wrong (/dev/fd instead of /proc/self/fd).

There's a comment in _posixsubprocess:

"NetBSD and OpenBSD have a /proc fs available (though not necessarily
mounted) and do not have fdescfs for /dev/fd."


Is this still valid?

At least on OpenBSD procfs have been removed: http://www.openbsd.org/faq/current.html#20140908

I'm not going to bother setting up a VM with an esoteric OS in it myself, if someone knows the past and current state of various OpenBSD versions and what to do there, feel free to commit OpenBSD conditional compilation bits as you see fit to make this do the right thing. :)

Indeed, I think my patch is not right as /dev/fd on OpenBSD is static.
I will continue to investigate.

The problem comes from safe_get_max_fd which return a too low value because of a bug in sysconf on OpenBSD [1]:

The value for _SC_STREAM_MAX is a minimum maximum, and required to be the same as ANSI C's FOPEN_MAX, so the returned value is a ridiculously small and misleading number.

[1] http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man3/sysconf.3

Here is a patch that uses getrlimit (that's works on OpenBSD) before using sysconf.

Unfortunately I don't have an OpenBSD install either.  From the
sysconf.c source ...

  http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/lib/libc/gen/sysconf.c?rev=1.22&content-type=text/plain


... it seems that sysconf(_SC_OPEN_MAX) also calls getrlimit().

But sysconf(_SC_OPEN_MAX) uses rlim_cur which is too low instead of rlim_max. My proposal is indeed describe in msg219477, it is not prefect but at least better than the current one for OpenBSD.

Correctly cast to long instead of int.

Tested on OpenBSD 5.6/amd64. lgtm.

getrlimit() is not an async-signal-safe function according to http://pubs.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html so you cannot call it from safe_get_max_fd().

having the getrlimit call done prior to the fork and using the value returned by that iff neither of the other two methods (fcntl and sysconf) are available or produced results seems like the best you can do.

also, rlim_t is an unsigned value yet the existing code in this module is using signed values for the file descriptors.  realistically i do not expect an rlim_t for max file descriptors to ever be > MAX_LONG as many file descriptor API calls require signed values for fds in order to use -1 as an error.

But checking the value for overflow and against the RLIM constants mentioned in http://pubs.opengroup.org/onlinepubs/009695399/basedefs/sys/resource.h.html before casting it via (long) seems pedantically wise.

Given that OpenBSD returns *bad* data via sysconf(3), I'm not sure that there's a good way to validate other than *only* calling getrlimit(3) on OpenBSD.

Is that an acceptable approach?

yeah, that's fine.  just surround the call to getrlimit with appropriate openbsd ifdef's and a comment.  it is _probably_ async signal safe given the nature of the function in most implementations even though it isn't on the official posix list (many things are).

I've updated Cédric's patch to only run that portion on OpenBSD.

Updated the patch based on review

New changeset 7df280b311d0 by Gregory P. Smith in branch '3.4':
Fix computation of max_fd on OpenBSD.  Issue #23852.
https://hg.python.org/cpython/rev/7df280b311d0

New changeset 08d0cc23fb00 by Gregory P. Smith in branch 'default':
Fix computation of max_fd on OpenBSD.  Issue #23852.
https://hg.python.org/cpython/rev/08d0cc23fb00