sources: 2.6rc1 tarball
system: FreeBSD 6.3 i386, gcc 3.4.6 (system compiler)
build:  ./configure; make

When run as part of the regression suite, test_signal fails.  When run
on its own, it passes.

In the failure case, the runtime of the test (as part of the whole test
run) appears to be several minutes, but when run on its own it completes
in < 10s.

This will require some random order regression runs to try and identify
the prior test which is provoking whatever is going wrong.

This issue has been around for several months at least, as I first
became aware of it with a checkout of r63892 on both FreeBSD 6.3 [gcc
3.4.6] and 7.0 [gcc 4.2.1] for both i386 and amd64 environments, but I
haven't had the opportunity to do the followup checking.

Can you check if it is not the itimer tests that are causing that ? I'm
interested in gathering some more info about the problem and hopefully
fixing it now. Take a look into issue2240 and verify if it is related to
your problem, please.

It doesn't appear to me to be related to issue 2240, as my build linked
against libpthread:

$ ldd ./python
./python:
        libutil.so.5 => /lib/libutil.so.5 (0x28187000)
        libm.so.4 => /lib/libm.so.4 (0x28193000)
        libpthread.so.2 => /lib/libpthread.so.2 (0x281a9000)
        libc.so.6 => /lib/libc.so.6 (0x281ce000)

I've had a chance to do some testing and it _is_ related to the itimer
tests (in test_wait4).

This is with r66550:

$ ./python -E -tt ./Lib/test/regrtest.py -l -v test_wait4 test_signal
test_wait4
test_wait (test.test_wait4.Wait4Test) ... ok

----------------------------------------------------------------------
Ran 1 test in 5.007s

OK
test_signal
test_getsignal (test.test_signal.BasicSignalTests) ... ok
test_out_of_range_signal_number_raises_error
(test.test_signal.BasicSignalTests) ... ok
test_setting_signal_handler_to_none_raises_error
(test.test_signal.BasicSignalTests) ... ok
test_main (test.test_signal.InterProcessSignalTests) ... FAIL
test_wakeup_fd_during (test.test_signal.WakeupSignalTests) ... ok
test_wakeup_fd_early (test.test_signal.WakeupSignalTests) ... ok
test_siginterrupt_off (test.test_signal.SiginterruptTest) ... ok
test_siginterrupt_on (test.test_signal.SiginterruptTest) ... ok
test_without_siginterrupt (test.test_signal.SiginterruptTest) ... ok
test_itimer_exc (test.test_signal.ItimerTest) ... ok
test_itimer_prof (test.test_signal.ItimerTest) ...

Running only test_signal:
$ ./python -E -tt ./Lib/test/regrtest.py -l -v test_signal
test_signal
test_getsignal (test.test_signal.BasicSignalTests) ... ok
test_out_of_range_signal_number_raises_error
(test.test_signal.BasicSignalTests)
 ... ok
test_setting_signal_handler_to_none_raises_error
(test.test_signal.BasicSignalTe
sts) ... ok
test_main (test.test_signal.InterProcessSignalTests) ... ok
test_wakeup_fd_during (test.test_signal.WakeupSignalTests) ... ok
test_wakeup_fd_early (test.test_signal.WakeupSignalTests) ... ok
test_siginterrupt_off (test.test_signal.SiginterruptTest) ... ok
test_siginterrupt_on (test.test_signal.SiginterruptTest) ... ok
test_without_siginterrupt (test.test_signal.SiginterruptTest) ... ok
test_itimer_exc (test.test_signal.ItimerTest) ... ok
test_itimer_prof (test.test_signal.ItimerTest) ... ('SIGPROF handler
invoked', (
27, <frame object at 0x83c860c>))
ok
test_itimer_real (test.test_signal.ItimerTest) ...
call pause()...
('SIGALRM handler invoked', (14, <frame object at 0x83c8c0c>))
ok
test_itimer_virtual (test.test_signal.ItimerTest) ... ('SIGVTALRM
handler invoke
d', (26, <frame object at 0x83c920c>))
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
last SIGVTALRM handler call
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
ok

----------------------------------------------------------------------
Ran 13 tests in 6.534s

OK
1 test OK.

Noting the interprocess signal test failure, I tried deactivating it and
the itimer tests still go off into the never never (using all available
CPU cycles too).

Oops - the itimer tests are in test_signal, not test_wait4.

test_wait4 just triggers the problems in test_signal (both the itimer
problems and the interprocess signal test failure).

After perusing test_wait4, I tried substituting test_fork1 for
test_wait4 and got the same behaviour from test_signal.

I compiled the C test case from issue 2240:

$ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -o test_2240
test_2240.c

{lifted as many gcc options off the standard Python compile as possible}

$ ldd test_2240
test_2240:
        libpthread.so.2 => /lib/libpthread.so.2 (0x2807a000)
        libc.so.6 => /lib/libc.so.6 (0x2809f000)

$ ./test_2240
0
1 0
deactive ITIMER_PROF

Spelunking with test_fork1, it seems that the interprocess signal test
failure is due to the HUP signal not being delivered from the subprocess
to the parent (line 99 of test_signal.py: self.assertTrue(self.a_called).

When you say "interprocess signal test" do you actually mean ItimerTest
? Because I don't see the former failing, and the later hangs because
signals are not being delivered to it (SIGVTALRM neither SIGPROF).

From the problematic test run log:

...
test_main (test.test_signal.InterProcessSignalTests) ... FAIL
...

I should be using the full name, sorry.

This failure seems unrelated to the itimer problem though (which is in
itimer_test_prof).  If I deactivate it, the itimer tests still goes into
the never never.

Both however are reliably triggered by running test_fork1 or test_wait4
(which uses the same machinery as test_fork1) before test_signal.

I should be more specific:
itimer_test_prof (test.test_signal.ItimerTest) appears to go into an
infinite loop when run after test_fork1 or test_wait4 have been run.

Last time I checked many more would cause ItimerTest to not run
properly, these were: test_asynchat, test_asyncore, test_decimal,
text_docxmlrpc, test_ftplib, test_logging, test_poplib, test_queue,
test_smtplib, test_socket and all these tests had in common the
threading.Event usage.

It is possible that I missed other tests, but what I concluded is that
freebsd, threads, and itimer doesn't work together :/

The tests are passing on FreeBSD 7.0 (only checked amd64 at this point).

I came across a reference to an errata notice for FreeBSD 6.x which
appears pertinent:
http://security.freebsd.org/advisories/FreeBSD-EN-08:01.libpthread.asc

As I read the above notice, the underlying issue is a FreeBSD bug which
will be fixed in FreeBSD 6.4 (expected to be released in the next couple
of months.

On this basis, I suggest closing this as "Won't fix".

Does anyone know what version of FreeBSD the FreeBSD buildslave is
running?  This problem is affecting most of its runs.

http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%20trunk

reports:
FreeBSD 6.2-RELEASE (VMWare Image, 256MB, 10GB on P4/1.8GHz Host)

Since this is apparently a known FreeBSD bug, we should put a
conditional skip into test_signal.

I'm hoping (though it's only a faint hope) that the change in r75986 might 
allow the test suite to run to completion on the FreeBSD buildslave.

Mark, can this bug be closed, then?

Well, test_signal is still failing.  So I guess this should stay open 
until we deal with the failure one way or another.  (E.g. skipping the 
test, or marking the failure as expected, or ...)

Put in a platform+release specific skip, perhaps.

The FreeBSD bot in question is now running 6.4, and test_signal is still failing.  I have access to a 6.3 box, which shows exactly the same failures as I see in the buildbot log.  On that box, doing

    t = threading.Thread(target=lambda: None)
    t.start()

with or without a t.join() afterward is enough to trigger the failures in test_signal.

Although it does not appear to have been fixed in 6.4, the 7.2 buildbot does not show the same problems.  (It has a somewhat different problem with *one* of the three failing tests in this case, but likely it is a different bug from this one, since the output is different.)  So I conclude that it is still most likely a platform problem, and I've added freebsd6 specific skips for those three tests in r80144.

Skips ported to 2.6 in r80295, py3k in r80296, and 3.1 in r80297.

On 2.6, the tests aren't skipped if test_support.verbose is 0.

David, if the fix looks good to you I can apply it.


test_itimer_virtual: timeout: likely cause: machine too slow or load too high.
test test_signal failed -- Traceback (most recent call last):
  File "/usr/home/db3l/buildarea/2.6.bolen-freebsd/build/Lib/test/test_signal.py", line 183, in test_main
    self.fail(tb)
AssertionError: Traceback (most recent call last):
  File "/usr/home/db3l/buildarea/2.6.bolen-freebsd/build/Lib/test/test_signal.py", line 166, in test_main
    self.run_test()
  File "/usr/home/db3l/buildarea/2.6.bolen-freebsd/build/Lib/test/test_signal.py", line 99, in run_test
    self.assertTrue(self.a_called)
  File "/usr/home/db3l/buildarea/2.6.bolen-freebsd/build/Lib/unittest.py", line 325, in failUnless
    if not expr: raise self.failureException, msg
AssertionError

I can't believe I messed that up.  Yes, the patch looks fine, please apply.

Thanks for looking at the patch! Committed in r81676.