Hi,

During the regress tests on OpenBSD, test_posix.test_fdlistdir() segfault.

Details:

$ ./python ./Tools/scripts/run_tests.py test_posix
/home/remi/dev/cpython_test/python -W default -bb -E -m test -r -w -j 0 -u all,-largefile,-network,-urlfetch,-audio,-gui test_posix
Using random seed 7449086
[1/1] test_posix
Fatal Python error: Segmentation fault

Current thread 0x0000000209419000:
  File "/home/remi/dev/cpython_test/Lib/test/test_posix.py", line 456 in test_fdlistdir
  File "/home/remi/dev/cpython_test/Lib/unittest/case.py", line 386 in _executeTestPart
  File "/home/remi/dev/cpython_test/Lib/unittest/case.py", line 441 in run
  File "/home/remi/dev/cpython_test/Lib/unittest/case.py", line 493 in __call__
  File "/home/remi/dev/cpython_test/Lib/unittest/suite.py", line 105 in run
  File "/home/remi/dev/cpython_test/Lib/unittest/suite.py", line 67 in __call__
  File "/home/remi/dev/cpython_test/Lib/unittest/suite.py", line 105 in run
  File "/home/remi/dev/cpython_test/Lib/unittest/suite.py", line 67 in __call__
  File "/home/remi/dev/cpython_test/Lib/test/support.py", line 1192 in run
  File "/home/remi/dev/cpython_test/Lib/test/support.py", line 1293 in _run_suite
  File "/home/remi/dev/cpython_test/Lib/test/support.py", line 1327 in run_unittest
  File "/home/remi/dev/cpython_test/Lib/test/test_posix.py", line 1022 in test_main
  File "/home/remi/dev/cpython_test/Lib/test/regrtest.py", line 1139 in runtest_inner
  File "/home/remi/dev/cpython_test/Lib/test/regrtest.py", line 915 in runtest
  File "/home/remi/dev/cpython_test/Lib/test/regrtest.py", line 439 in main
  File "/home/remi/dev/cpython_test/Lib/test/regrtest.py", line 1717 in <module>
  File "/home/remi/dev/cpython_test/Lib/runpy.py", line 73 in _run_code
  File "/home/remi/dev/cpython_test/Lib/runpy.py", line 160 in _run_module_as_main
Traceback (most recent call last):
  File "/home/remi/dev/cpython_test/Lib/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/remi/dev/cpython_test/Lib/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/home/remi/dev/cpython_test/Lib/test/__main__.py", line 13, in <module>
    regrtest.main()
  File "/home/remi/dev/cpython_test/Lib/test/regrtest.py", line 683, in main
    raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_posix: Exit code -11
[98138 refs]

$ ./python Lib/test/test_posix.py
testNoArgFunctions (__main__.PosixTester) ... ok
test_access (__main__.PosixTester) ... ok
test_chdir (__main__.PosixTester) ... ok
test_chflags (__main__.PosixTester) ... ok
test_chown (__main__.PosixTester) ... ok
test_confstr (__main__.PosixTester) ... ok
test_cpu_set_basic (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_bitwise (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_cmp (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_dup (__main__.PosixTester) ... ok
test_dup2 (__main__.PosixTester) ... ok
test_environ (__main__.PosixTester) ... ok
test_faccessat (__main__.PosixTester) ... ok
test_fchmodat (__main__.PosixTester) ... ok
test_fchown (__main__.PosixTester) ... ok
test_fchownat (__main__.PosixTester) ... ok
test_fdlistdir (__main__.PosixTester) ... zsh: segmentation fault (core dumped)  ./python Lib/test/test_posix.py

########
with gdb
########

$ gdb ./python
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-unknown-openbsd5.0"...
(gdb) run Lib/test/test_posix.py
Starting program: /home/remi/dev/cpython_test/python Lib/test/test_posix.py
testNoArgFunctions (__main__.PosixTester) ... ok
test_access (__main__.PosixTester) ... ok
test_chdir (__main__.PosixTester) ... ok
test_chflags (__main__.PosixTester) ... ok
test_chown (__main__.PosixTester) ... ok
test_confstr (__main__.PosixTester) ... ok
test_cpu_set_basic (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_bitwise (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_cmp (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_dup (__main__.PosixTester) ... ok
test_dup2 (__main__.PosixTester) ... ok
test_environ (__main__.PosixTester) ... ok
test_faccessat (__main__.PosixTester) ... ok
test_fchmodat (__main__.PosixTester) ... ok
test_fchown (__main__.PosixTester) ... ok
test_fchownat (__main__.PosixTester) ... ok
test_fdlistdir (__main__.PosixTester) ...
Program received signal SIGSEGV, Segmentation fault.
[Switching to process 21658, thread 0x20a519000]
_readdir_unlocked (dirp=0xafb0e80, result=0x7f7ffffd7ac0, skipdeleted=1)
    at /usr/src/lib/libc/gen/readdir.c:44
44                      if (dirp->dd_loc >= dirp->dd_size)
(gdb) bt
#0  _readdir_unlocked (dirp=0xafb0e80, result=0x7f7ffffd7ac0, skipdeleted=1)
    at /usr/src/lib/libc/gen/readdir.c:44
#1  0x000000020f160f51 in readdir (dirp=0xafb0e80) at /usr/src/lib/libc/gen/readdir.c:80
#2  0x00000000004af33b in posix_fdlistdir (self=Variable "self" is not available.
) at ./Modules/posixmodule.c:2832
#3  0x00000000004694f4 in PyEval_EvalFrameEx (f=0x20973fc30, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4000
#4  0x000000000046a71c in PyEval_EvalFrameEx (f=0x208aa4c30, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4088
#5  0x000000000046b954 in PyEval_EvalCodeEx (_co=0x2052c2640, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#6  0x0000000000467d8b in PyEval_EvalFrameEx (f=0x20e52a830, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4098
#7  0x000000000046b954 in PyEval_EvalCodeEx (_co=0x2052c27c0, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#8  0x000000000051a243 in function_call (func=0x2024054b0, arg=0x20a882600, kw=0x20a8836c8)
    at Objects/funcobject.c:629
#9  0x00000000004ec5bf in PyObject_Call (func=0x2024054b0, arg=0x20a882600, kw=0x20a8836c8)
    at Objects/abstract.c:2149
#10 0x0000000000465936 in PyEval_EvalFrameEx (f=0x20a884660, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4315
#11 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x2052c2ac0, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#12 0x000000000051a243 in function_call (func=0x202405620, arg=0x20a882588, kw=0x0)
    at Objects/funcobject.c:629
#13 0x00000000004ec5bf in PyObject_Call (func=0x202405620, arg=0x20a882588, kw=0x0)
    at Objects/abstract.c:2149
#14 0x0000000000506ed9 in method_call (func=0x202405620, arg=0x20a87f7d0, kw=0x0)
    at Objects/classobject.c:318
#15 0x00000000004ec5bf in PyObject_Call (func=0x20bf0cba0, arg=0x20a87f7d0, kw=0x0)
    at Objects/abstract.c:2149
#16 0x000000000042a7e6 in slot_tp_call (self=Variable "self" is not available.
) at Objects/typeobject.c:5194
#17 0x00000000004ec5bf in PyObject_Call (func=0x20bf25a70, arg=0x20a87f7d0, kw=0x0)
    at Objects/abstract.c:2149
#18 0x00000000004614ef in PyEval_EvalFrameEx (f=0x20e52bc30, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4220
#19 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x20240e1c0, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#20 0x000000000051a243 in function_call (func=0x20240dec0, arg=0x20a882498, kw=0x20a883580)
    at Objects/funcobject.c:629
#21 0x00000000004ec5bf in PyObject_Call (func=0x20240dec0, arg=0x20a882498, kw=0x20a883580)
    at Objects/abstract.c:2149
#22 0x0000000000465936 in PyEval_EvalFrameEx (f=0x20a884260, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4315
#23 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x2023f5f40, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#24 0x000000000051a243 in function_call (func=0x20240dd50, arg=0x20bf0cab0, kw=0x0)
    at Objects/funcobject.c:629
#25 0x00000000004ec5bf in PyObject_Call (func=0x20240dd50, arg=0x20bf0cab0, kw=0x0)
    at Objects/abstract.c:2149
#26 0x0000000000506ed9 in method_call (func=0x20240dd50, arg=0x20bf25290, kw=0x0)
    at Objects/classobject.c:318
#27 0x00000000004ec5bf in PyObject_Call (func=0x20bf0cb28, arg=0x20bf25290, kw=0x0)
    at Objects/abstract.c:2149
#28 0x000000000042a7e6 in slot_tp_call (self=Variable "self" is not available.
) at Objects/typeobject.c:5194
#29 0x00000000004ec5bf in PyObject_Call (func=0x20bf25300, arg=0x20bf25290, kw=0x0)
    at Objects/abstract.c:2149
#30 0x00000000004614ef in PyEval_EvalFrameEx (f=0x20e52c830, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4220
#31 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x20240e1c0, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#32 0x000000000051a243 in function_call (func=0x20240dec0, arg=0x20bf0c9c0, kw=0x20a883438)
    at Objects/funcobject.c:629
#33 0x00000000004ec5bf in PyObject_Call (func=0x20240dec0, arg=0x20bf0c9c0, kw=0x20a883438)
    at Objects/abstract.c:2149
#34 0x0000000000465936 in PyEval_EvalFrameEx (f=0x20a884060, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4315
#35 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x2023f5f40, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#36 0x000000000051a243 in function_call (func=0x20240dd50, arg=0x20bf0c948, kw=0x0)
    at Objects/funcobject.c:629
#37 0x00000000004ec5bf in PyObject_Call (func=0x20240dd50, arg=0x20bf0c948, kw=0x0)
    at Objects/abstract.c:2149
#38 0x0000000000506ed9 in method_call (func=0x20240dd50, arg=0x20bf05fb0, kw=0x0)
---Type <return> to continue, or q <return> to quit---
    at Objects/classobject.c:318
#39 0x00000000004ec5bf in PyObject_Call (func=0x2049d8150, arg=0x20bf05fb0, kw=0x0)
    at Objects/abstract.c:2149
#40 0x000000000042a7e6 in slot_tp_call (self=Variable "self" is not available.
) at Objects/typeobject.c:5194
#41 0x00000000004ec5bf in PyObject_Call (func=0x20bf05df0, arg=0x20bf05fb0, kw=0x0)
    at Objects/abstract.c:2149
#42 0x00000000004614ef in PyEval_EvalFrameEx (f=0x2015e2c30, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4220
#43 0x000000000046a71c in PyEval_EvalFrameEx (f=0x20bea9830, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4088
#44 0x000000000046a71c in PyEval_EvalFrameEx (f=0x20f636030, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4088
#45 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x209f27400, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#46 0x0000000000467d8b in PyEval_EvalFrameEx (f=0x20afb5c30, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4098
#47 0x000000000046a71c in PyEval_EvalFrameEx (f=0x20cde2df0, throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:4088
#48 0x000000000046b954 in PyEval_EvalCodeEx (_co=0x208dc1b80, globals=Variable "globals" is not available.
) at Python/ceval.c:3375
#49 0x000000000046babb in PyEval_EvalCode (co=Variable "co" is not available.
) at Python/ceval.c:770
#50 0x000000000048d3e7 in run_mod (mod=Variable "mod" is not available.
) at Python/pythonrun.c:1793
#51 0x000000000048d4ee in PyRun_FileExFlags (fp=0x20f5dd8e0,
    filename=0x209f123b0 "Lib/test/test_posix.py", start=257, globals=0x2073b0580,
    locals=0x2073b0580, closeit=1, flags=0x7f7ffffdac50) at Python/pythonrun.c:1750
#52 0x00000000004900bf in PyRun_SimpleFileExFlags (fp=0x20f5dd8e0,
    filename=0x209f123b0 "Lib/test/test_posix.py", closeit=1, flags=0x7f7ffffdac50)
    at Python/pythonrun.c:1275
#53 0x00000000004a2990 in Py_Main (argc=Variable "argc" is not available.
) at Modules/main.c:299
#54 0x0000000000415cba in main (argc=2, argv=0x7f7ffffdad18) at ./Modules/python.c:59


Don't hesitate to ask if you need more information.
Thanks a lot for your help.

Cheers,
Remi.

Does it always segfault?

Try:
./python -c 'import os; print(os.fdlistdir(os.open("/tmp", os.O_RDONLY)))'

with various values for /tmp.

From what I can see, the code for fdlistdir is basically the same as os.listdir().
If possible, try os.listdir() as well.

Hi,

thanks for your response.

Yes it always segfault:

$ ./python -c 'import os; print(os.fdlistdir(os.open("/tmp", os.O_RDONLY)))'
zsh: segmentation fault (core dumped)  ./python -c 'import os; print(os.fdlistdir(os.open("/tmp", os.O_RDONLY)))'

$ ./python -c 'import os; print(os.fdlistdir(os.open("/tmp/test_python", os.O_RDONLY)))'
zsh: segmentation fault (core dumped)  ./python -c 

$ ./python -c 'import os; print(os.fdlistdir(os.open("/var/tmp", os.O_RDONLY)))'
zsh: segmentation fault (core dumped)  ./python -c 'import os; print(os.fdlistdir(os.open("/var/tmp", os.O_RDONLY)))

With listdir it correctly works:

$ ./python -c 'import os; print(os.listdir("/tmp"))'                          
['.X11-unix', '.ICE-unix', 'test_python', 'tmpl80ewt', 'tmpled292']
[42658 refs]

$ ./python -c 'import os; print(os.listdir("/tmp/test_python/"))'
['1file', '2file', '3file']
[42658 refs]

$ ./python -c 'import os; print(os.listdir("/var/tmp/"))'        
['vi.recover', 'kdecache-remi']
[42658 refs]

Remi.

It looks like a kernel bug !? Are you able to write a C script reproducing the problem? If not, I can try to write it.

> It looks like a kernel bug !?

That's what I thought given that it appears to be working on all the other platforms.

Hi,

I tested with this program in C:

#include <stdio.h>
#include <dirent.h>
#include <fcntl.h>
#include <stdlib.h>

int
main(void)
{
    DIR *d;
    struct dirent *dp;
    int dfd;

    if ((d = fdopendir((dfd = open("/tmp", O_RDONLY)))) == NULL) {
        fprintf(stderr, "Cannot open /tmp directory\n");
        exit(1);
    }
    while ((dp = readdir(d)) != NULL) {
        if (dp->d_name[0] == '.')
            continue;
            printf("%s, \n", dp->d_name);
    }
    closedir(d);
    return 0;
}


and it seems to correctly works.

Remi.

I'm not sure what the status of Python and OpenBSD support is but I just tried the latest stable version of OpenBSD (4.9) in VirtualBox and it won't compile fully. It segfaults while trying to run setup.py (I think).

I see you're running OpenBSD 5.0. Does it compile easily on that without having to jump through hoops?

> It looks like a kernel bug !?

And you know what? fdopendir() function has been introducted in OpenBSD 5.0!

[Switching to process 21658, thread 0x20a519000]
_readdir_unlocked (dirp=0xafb0e80, result=0x7f7ffffd7ac0, skipdeleted=1)
    at /usr/src/lib/libc/gen/readdir.c:44
44                      if (dirp->dd_loc >= dirp->dd_size)


Looks like dirp points to an invali location in memory.
Could you try display it ("p *dirp")?

But this definitely looks like a kernel/libc bug...

I think that the problem is that fdopendir() is not defined. If a function is not defined, C uses int as the result type. An int is not enough to store a 64-bit pointer. See in gdb output: dirp is 0x0afb0e80 whereas other pointers look like 0x20973fc30. You missed the highest hexa digit (0x2).

fdopendir() requires IEEE 1003.1-2008 and so "#define _POSIX_C_SOURCE 200809L", whereas pyconfig.h defines _POSIX_C_SOURCE to 200112L (POSIX 2001).

Something should be done in configure.in, near:
-----------------------------------------
case $ac_sys_system/$ac_sys_release in
  # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined,
  # even though select is a POSIX function. Reported by J. Ribbens.
  # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish.
  # In addition, Stefan Krah confirms that issue #1244610 exists through
  # OpenBSD 4.6, but is fixed in 4.7.
  OpenBSD/2.* | OpenBSD/3.* | OpenBSD/4.@<:@0123456@:>@) 
    define_xopen_source=no
    # OpenBSD undoes our definition of __BSD_VISIBLE if _XOPEN_SOURCE is
    # also defined. This can be overridden by defining _BSD_SOURCE
    # As this has a different meaning on Linux, only define it on OpenBSD
    AC_DEFINE(_BSD_SOURCE, 1, [Define on OpenBSD to activate all library features])
    ;;
  OpenBSD/*)
    # OpenBSD undoes our definition of __BSD_VISIBLE if _XOPEN_SOURCE is
    # also defined. This can be overridden by defining _BSD_SOURCE
    # As this has a different meaning on Linux, only define it on OpenBSD
    AC_DEFINE(_BSD_SOURCE, 1, [Define on OpenBSD to activate all library features])
    ;;
-----------------------------------------

or maybe in

-----------------------------------------
if test $define_xopen_source = yes
then
  AC_DEFINE(_XOPEN_SOURCE, 600, 
            Define to the level of X/Open that your system supports)

  # On Tru64 Unix 4.0F, defining _XOPEN_SOURCE also requires
  # definition of _XOPEN_SOURCE_EXTENDED and _POSIX_C_SOURCE, or else
  # several APIs are not declared. Since this is also needed in some
  # cases for HP-UX, we define it globally.
  AC_DEFINE(_XOPEN_SOURCE_EXTENDED, 1,
   	    Define to activate Unix95-and-earlier features)

  AC_DEFINE(_POSIX_C_SOURCE, 200112L, Define to activate features from IEEE Stds 1003.1-2001)  
fi
-----------------------------------------

I tried "AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE Stds 1003.1-2008)" but it doesn't work.

Add "#define _POSIX_C_SOURCE 200809L" at the beginning of Modules/posixmodule.c does works around this issue.

> I think that the problem is that fdopendir() is not defined. If a function is not defined, C uses int as the result type. An int is not enough to store a 64-bit pointer. See in gdb output: dirp is 0x0afb0e80 whereas other pointers look like 0x20973fc30. You missed the highest hexa digit (0x2).

Yeah, I noticed that. I didn't make the connection with the
possibility of missing prototype though. Nice catch.

> I tried "AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE Stds 1003.1-2008)" but it doesn't work.
>

You mean that the patch you attached doesn't work, correct?
I know it's a stupid question, but you're sure you didn't forget to
run autoconf/autoheader?

>> I tried "AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE Stds 1003.1-2008)" but it doesn't work.
>
> You mean that the patch you attached doesn't work, correct?

I ran autoconf, re-ran configure and it doesn't work. You can test 
without touching configure.in: edit pyconfig.h directly.

On Linux, all "features" flags are defined/undefined in 
/usr/include/features.h. Depending on _POSIX_C_SOURCE, _XOPEN_SOURCE, 
_BSD_SOURCE, ..., you get a different POSIX level.

For example:

/* If _GNU_SOURCE was defined by the user, turn on all the other 
features.  */
#ifdef _GNU_SOURCE
# undef  _ISOC99_SOURCE
# define _ISOC99_SOURCE	1
# undef  _POSIX_SOURCE
# define _POSIX_SOURCE	1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE	200809L
# undef  _XOPEN_SOURCE
# define _XOPEN_SOURCE	700
# ...
#endif

I suppose that there is a conflict between Python's _POSIX_C_SOURCE and 
other defines related to the POSIX level.

Hi,

this is the result of gcc -E on Modules/posixmodule.o, asked by haypo.

Thanks for your help,

Cheers,
Remi.

Rémi : what happens if you manually add

    extern DIR *fdopendir(int fd);

at the beginning of posixmodule.c?

> Rémi : what happens if you manually add
> extern DIR *fdopendir(int fd);
> at the beginning of posixmodule.c?

Hello,

it seems to correctly work:
 ./python Lib/test/test_posix.py 
testNoArgFunctions (__main__.PosixTester) ... ok
test_access (__main__.PosixTester) ... ok
test_chdir (__main__.PosixTester) ... ok
test_chflags (__main__.PosixTester) ... ok
test_chown (__main__.PosixTester) ... ok
test_confstr (__main__.PosixTester) ... ok
test_cpu_set_basic (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_bitwise (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_cpu_set_cmp (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_dup (__main__.PosixTester) ... ok
test_dup2 (__main__.PosixTester) ... ok
test_environ (__main__.PosixTester) ... ok
test_faccessat (__main__.PosixTester) ... ok
test_fchmodat (__main__.PosixTester) ... ok
test_fchown (__main__.PosixTester) ... ok
test_fchownat (__main__.PosixTester) ... ok
test_fdlistdir (__main__.PosixTester) ... ok
test_fexecve (__main__.PosixTester) ... skipped 'test needs posix.fexecve()'
test_fstat (__main__.PosixTester) ... ok
test_fstatat (__main__.PosixTester) ... ok
test_fstatvfs (__main__.PosixTester) ... ok
test_ftruncate (__main__.PosixTester) ... ok
test_futimens (__main__.PosixTester) ... ok
test_futimes (__main__.PosixTester) ... ok
test_futimesat (__main__.PosixTester) ... skipped 'test needs posix.futimesat()'
test_get_and_set_scheduler_and_param (__main__.PosixTester) ... skipped "can't change scheduler"
test_getcwd_long_pathnames (__main__.PosixTester) ... ok
test_getgrouplist (__main__.PosixTester) ... ok
test_getgroups (__main__.PosixTester) ... ok
test_getresgid (__main__.PosixTester) ... ok
test_getresuid (__main__.PosixTester) ... ok
test_initgroups (__main__.PosixTester) ... ok
test_lchflags_regular_file (__main__.PosixTester) ... skipped 'test needs os.lchflags()'
test_lchflags_symlink (__main__.PosixTester) ... skipped 'test needs os.lchflags()'
test_lchown (__main__.PosixTester) ... ok
test_linkat (__main__.PosixTester) ... ok
test_listdir (__main__.PosixTester) ... ok
test_listdir_default (__main__.PosixTester) ... ok
test_lockf (__main__.PosixTester) ... ok
test_lutimes (__main__.PosixTester) ... skipped 'test needs posix.lutimes()'
test_mkdirat (__main__.PosixTester) ... ok
test_mkfifo (__main__.PosixTester) ... ok
test_mkfifoat (__main__.PosixTester) ... ok
test_mknod (__main__.PosixTester) ... ok
test_mknodat (__main__.PosixTester) ... ok
test_openat (__main__.PosixTester) ... ok
test_oscloexec (__main__.PosixTester) ... ok
test_osexlock (__main__.PosixTester) ... ok
test_osshlock (__main__.PosixTester) ... ok
test_pipe (__main__.PosixTester) ... ok
test_pipe2 (__main__.PosixTester) ... skipped 'test needs os.pipe2()'
test_posix_fadvise (__main__.PosixTester) ... skipped 'test needs posix.posix_fadvise()'
test_posix_fallocate (__main__.PosixTester) ... skipped 'test needs posix.posix_fallocate()'
test_pread (__main__.PosixTester) ... ok
test_pwrite (__main__.PosixTester) ... ok
test_readlinkat (__main__.PosixTester) ... ok
test_readv (__main__.PosixTester) ... ok
test_renameat (__main__.PosixTester) ... ok
test_sched_affinity (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_sched_priority (__main__.PosixTester) ... ok
test_sched_rr_get_interval (__main__.PosixTester) ... skipped 'no function'
test_sched_yield (__main__.PosixTester) ... ok
test_setresgid (__main__.PosixTester) ... ok
test_setresgid_exception (__main__.PosixTester) ... ok
test_setresuid (__main__.PosixTester) ... ok
test_setresuid_exception (__main__.PosixTester) ... ok
test_stat (__main__.PosixTester) ... ok
test_statvfs (__main__.PosixTester) ... ok
test_strerror (__main__.PosixTester) ... ok
test_symlinkat (__main__.PosixTester) ... ok
test_truncate (__main__.PosixTester) ... ok
test_umask (__main__.PosixTester) ... ok
test_unlinkat (__main__.PosixTester) ... ok
test_utime (__main__.PosixTester) ... ok
test_utimensat (__main__.PosixTester) ... ok
test_waitid (__main__.PosixTester) ... skipped 'test needs posix.waitid()'
test_writev (__main__.PosixTester) ... ok
test_initgroups (__main__.PosixGroupsTester) ... skipped 'not enough privileges'
test_setgroups (__main__.PosixGroupsTester) ... skipped 'not enough privileges'

----------------------------------------------------------------------
Ran 79 tests in 0.801s

OK (skipped=17)

And this command now works:

$ ./python -c 'import os; print(os.fdlistdir(os.open("/tmp", os.O_RDONLY)))'
['.X11-unix', '.ICE-unix', 'orbit-remi', '.X0-lock', 'dbus-WWfX2JhDtb']

Attached file patch-Modules_posixmodule_c is the patch to do this.

Thanks a lot,

Remi.

> And this command now works:
> 
> $ ./python -c 'import os; print(os.fdlistdir(os.open("/tmp", os.O_RDONLY)))'
> ['.X11-unix', '.ICE-unix', 'orbit-remi', '.X0-lock', 'dbus-WWfX2JhDtb']
> 
> Attached file patch-Modules_posixmodule_c is the patch to do this.

Thank you.
It would be nice if you could find out in which header file fdopendir()
is defined. A recursive grep in /usr/include (?) should do the trick, I
think.

> It would be nice if you could find out in which header file fdopendir() is defined.

Hi,

fdopendir() is in /usr/include/dirent.h:

#if __POSIX_VISIBLE >= 200809
DIR *fdopendir(int);
#endif

You could find the file dirent.h here: http://www.openbsd.org/cgi-bin/cvsweb/src/include/dirent.h.

Thanks,

Remi.

> this is the result of gcc -E on Modules/posixmodule.o, asked by haypo.

And this confirms that __POSIX_VISIBLE < 200809 when <dirent.h> is included, hence the missing prototype.

> I suppose that there is a conflict between Python's _POSIX_C_SOURCE and 
> other defines related to the POSIX level.

Lookie here:
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/sys/cdefs.h?rev=1.31
"""
#ifdef _XOPEN_SOURCE
# if (_XOPEN_SOURCE - 0 >= 700)
#  define __XPG_VISIBLE         700
#  undef _POSIX_C_SOURCE
#  define _POSIX_C_SOURCE       200809L
# elif (_XOPEN_SOURCE - 0 >= 600)
#  define __XPG_VISIBLE         600
#  undef _POSIX_C_SOURCE
#  define _POSIX_C_SOURCE       200112L
"""

configure.in defines _XOPEN_SOURCE to 600:
if test $define_xopen_source = yes
then
  AC_DEFINE(_XOPEN_SOURCE, 600,
            Define to the level of X/Open that your system supports)


So, try with _POSIX_C_SOURCE set to 200809L and _XOPEN_SOURCE to 700 (see http://pubs.opengroup.org/onlinepubs/9699919799/), and it should work.

New changeset cf66578d03d1 by Victor Stinner in branch 'default':
Issue #12852: Set _XOPEN_SOURCE to 700 to get POSIX 2008
http://hg.python.org/cpython/rev/cf66578d03d1

> try with _POSIX_C_SOURCE set to 200809L and _XOPEN_SOURCE to 700

_POSIX_C_SOURCE value is set automatically depending on _XOPEN_SOURCE value. I chose to set _XOPEN_SOURCE to 700, instead of 600: it works on OpenBSD 5.0. I don't know if _XOPEN_SOURCE=700 is supported by old platforms.

I don't want to touch such sensitive thing like configure in stable releases.

@rpointel: You can backport manually the fix into OpenBSD for Python 3.2, or (safer?) patch pyconfig.h to set _XOPEN_SOURCE to 700.

According to configure.in, it was not possible to set _XOPEN_SOURCE on OpenBSD < 4.7:

  # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined,
  # even though select is a POSIX function. Reported by J. Ribbens.
  # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish.
  # In addition, Stefan Krah confirms that issue #1244610 exists through
  # OpenBSD 4.6, but is fixed in 4.7.

@rpointel: Does the fix work for you? If yes, I will close the issue.

> _POSIX_C_SOURCE value is set automatically depending on _XOPEN_SOURCE
> value.

I know, but I think it's better to be consistent an also bump _POSIX_C_SOURCE to POSIX 2008, and follow POSIX's recommandation (http://pubs.opengroup.org/onlinepubs/9699919799/):
"""
A Strictly Conforming POSIX Application is an application that requires only the facilities described in POSIX.1-2008. Such an application:
[...]
For the C programming language, shall define _POSIX_C_SOURCE to be 200809L before any header is included
"""

> @rpointel: Does the fix work for you? If yes, I will close the issue.

Hi, yes it sounds good for me:

$ hg pull -u
$ ./configure && make
$ ./python Lib/test/test_posix.py
[...]
test_fdlistdir (__main__.PosixTester) ... ok
[...]
Ran 79 tests in 0.097s
OK (skipped=17)

Thanks.

Remi.

New changeset 92842e347d98 by Victor Stinner in branch 'default':
Issue #12852: Set _POSIX_C_SOURCE to 200809 to get POSIX 2008
http://hg.python.org/cpython/rev/92842e347d98

> I know, but I think it's better to be consistent an also bump
> _POSIX_C_SOURCE to POSIX 2008

Oh, I missed AC_DEFINE(_POSIX_C_SOURCE, ...) after AC_DEFINE(_XOPEN_SOURCE, 
...). Fixed.