classification
Title: 26.rc1: test_signal issue on FreeBSD 6.3
Type: behavior Stage: needs patch
Components: Tests Versions: Python 3.2, Python 3.1, Python 2.7, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: aimacintyre, eric.smith, gpolo, mark.dickinson, r.david.murray (5)
Priority: normal Keywords buildbot, easy

Created on 2008-09-14 11:41 by aimacintyre, last changed 2009-11-06 17:42 by r.david.murray.

Messages (20)
msg73212 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-14 11:41
sources: 2.6rc1 tarball
system: FreeBSD 6.3 i386, gcc 3.4.6 (system compiler)
build:  ./configure; make

When run as part of the regression suite, test_signal fails.  When run
on its own, it passes.

In the failure case, the runtime of the test (as part of the whole test
run) appears to be several minutes, but when run on its own it completes
in < 10s.

This will require some random order regression runs to try and identify
the prior test which is provoking whatever is going wrong.

This issue has been around for several months at least, as I first
became aware of it with a checkout of r63892 on both FreeBSD 6.3 [gcc
3.4.6] and 7.0 [gcc 4.2.1] for both i386 and amd64 environments, but I
haven't had the opportunity to do the followup checking.
msg73231 - (view) Author: Guilherme Polo (gpolo) Date: 2008-09-14 17:58
Can you check if it is not the itimer tests that are causing that ? I'm
interested in gathering some more info about the problem and hopefully
fixing it now. Take a look into issue2240 and verify if it is related to
your problem, please.
msg73253 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-15 10:32
It doesn't appear to me to be related to issue 2240, as my build linked
against libpthread:

$ ldd ./python
./python:
        libutil.so.5 => /lib/libutil.so.5 (0x28187000)
        libm.so.4 => /lib/libm.so.4 (0x28193000)
        libpthread.so.2 => /lib/libpthread.so.2 (0x281a9000)
        libc.so.6 => /lib/libc.so.6 (0x281ce000)
msg73625 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 11:48
I've had a chance to do some testing and it _is_ related to the itimer
tests (in test_wait4).

This is with r66550:

$ ./python -E -tt ./Lib/test/regrtest.py -l -v test_wait4 test_signal
test_wait4
test_wait (test.test_wait4.Wait4Test) ... ok

----------------------------------------------------------------------
Ran 1 test in 5.007s

OK
test_signal
test_getsignal (test.test_signal.BasicSignalTests) ... ok
test_out_of_range_signal_number_raises_error
(test.test_signal.BasicSignalTests) ... ok
test_setting_signal_handler_to_none_raises_error
(test.test_signal.BasicSignalTests) ... ok
test_main (test.test_signal.InterProcessSignalTests) ... FAIL
test_wakeup_fd_during (test.test_signal.WakeupSignalTests) ... ok
test_wakeup_fd_early (test.test_signal.WakeupSignalTests) ... ok
test_siginterrupt_off (test.test_signal.SiginterruptTest) ... ok
test_siginterrupt_on (test.test_signal.SiginterruptTest) ... ok
test_without_siginterrupt (test.test_signal.SiginterruptTest) ... ok
test_itimer_exc (test.test_signal.ItimerTest) ... ok
test_itimer_prof (test.test_signal.ItimerTest) ...

Running only test_signal:
$ ./python -E -tt ./Lib/test/regrtest.py -l -v test_signal
test_signal
test_getsignal (test.test_signal.BasicSignalTests) ... ok
test_out_of_range_signal_number_raises_error
(test.test_signal.BasicSignalTests)
 ... ok
test_setting_signal_handler_to_none_raises_error
(test.test_signal.BasicSignalTe
sts) ... ok
test_main (test.test_signal.InterProcessSignalTests) ... ok
test_wakeup_fd_during (test.test_signal.WakeupSignalTests) ... ok
test_wakeup_fd_early (test.test_signal.WakeupSignalTests) ... ok
test_siginterrupt_off (test.test_signal.SiginterruptTest) ... ok
test_siginterrupt_on (test.test_signal.SiginterruptTest) ... ok
test_without_siginterrupt (test.test_signal.SiginterruptTest) ... ok
test_itimer_exc (test.test_signal.ItimerTest) ... ok
test_itimer_prof (test.test_signal.ItimerTest) ... ('SIGPROF handler
invoked', (
27, <frame object at 0x83c860c>))
ok
test_itimer_real (test.test_signal.ItimerTest) ...
call pause()...
('SIGALRM handler invoked', (14, <frame object at 0x83c8c0c>))
ok
test_itimer_virtual (test.test_signal.ItimerTest) ... ('SIGVTALRM
handler invoke
d', (26, <frame object at 0x83c920c>))
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
last SIGVTALRM handler call
('SIGVTALRM handler invoked', (26, <frame object at 0x83c920c>))
ok

----------------------------------------------------------------------
Ran 13 tests in 6.534s

OK
1 test OK.

Noting the interprocess signal test failure, I tried deactivating it and
the itimer tests still go off into the never never (using all available
CPU cycles too).
msg73626 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 11:52
Oops - the itimer tests are in test_signal, not test_wait4.

test_wait4 just triggers the problems in test_signal (both the itimer
problems and the interprocess signal test failure).
msg73627 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 12:01
After perusing test_wait4, I tried substituting test_fork1 for
test_wait4 and got the same behaviour from test_signal.
msg73630 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 12:40
I compiled the C test case from issue 2240:

$ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -o test_2240
test_2240.c

{lifted as many gcc options off the standard Python compile as possible}

$ ldd test_2240
test_2240:
        libpthread.so.2 => /lib/libpthread.so.2 (0x2807a000)
        libc.so.6 => /lib/libc.so.6 (0x2809f000)

$ ./test_2240
0
1 0
deactive ITIMER_PROF
msg73632 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 12:47
Spelunking with test_fork1, it seems that the interprocess signal test
failure is due to the HUP signal not being delivered from the subprocess
to the parent (line 99 of test_signal.py: self.assertTrue(self.a_called).
msg73633 - (view) Author: Guilherme Polo (gpolo) Date: 2008-09-23 13:03
When you say "interprocess signal test" do you actually mean ItimerTest
? Because I don't see the former failing, and the later hangs because
signals are not being delivered to it (SIGVTALRM neither SIGPROF).
msg73636 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 13:27
From the problematic test run log:

...
test_main (test.test_signal.InterProcessSignalTests) ... FAIL
...

I should be using the full name, sorry.

This failure seems unrelated to the itimer problem though (which is in
itimer_test_prof).  If I deactivate it, the itimer tests still goes into
the never never.

Both however are reliably triggered by running test_fork1 or test_wait4
(which uses the same machinery as test_fork1) before test_signal.
msg73637 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-23 13:30
I should be more specific:
itimer_test_prof (test.test_signal.ItimerTest) appears to go into an
infinite loop when run after test_fork1 or test_wait4 have been run.
msg73638 - (view) Author: Guilherme Polo (gpolo) Date: 2008-09-23 13:32
Last time I checked many more would cause ItimerTest to not run
properly, these were: test_asynchat, test_asyncore, test_decimal,
text_docxmlrpc, test_ftplib, test_logging, test_poplib, test_queue,
test_smtplib, test_socket and all these tests had in common the
threading.Event usage.

It is possible that I missed other tests, but what I concluded is that
freebsd, threads, and itimer doesn't work together :/
msg73956 - (view) Author: Andrew I MacIntyre (aimacintyre) Date: 2008-09-28 06:46
The tests are passing on FreeBSD 7.0 (only checked amd64 at this point).

I came across a reference to an errata notice for FreeBSD 6.x which
appears pertinent:
http://security.freebsd.org/advisories/FreeBSD-EN-08:01.libpthread.asc

As I read the above notice, the underlying issue is a FreeBSD bug which
will be fixed in FreeBSD 6.4 (expected to be released in the next couple
of months.

On this basis, I suggest closing this as "Won't fix".
msg94721 - (view) Author: R. David Murray (r.david.murray) Date: 2009-10-30 20:01
Does anyone know what version of FreeBSD the FreeBSD buildslave is
running?  This problem is affecting most of its runs.
msg94722 - (view) Author: Eric Smith (eric.smith) Date: 2009-10-30 20:27
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%20trunk

reports:
FreeBSD 6.2-RELEASE (VMWare Image, 256MB, 10GB on P4/1.8GHz Host)
msg94724 - (view) Author: R. David Murray (r.david.murray) Date: 2009-10-30 20:48
Since this is apparently a known FreeBSD bug, we should put a
conditional skip into test_signal.
msg94753 - (view) Author: Mark Dickinson (mark.dickinson) Date: 2009-10-31 11:08
I'm hoping (though it's only a faint hope) that the change in r75986 might 
allow the test suite to run to completion on the FreeBSD buildslave.
msg94984 - (view) Author: R. David Murray (r.david.murray) Date: 2009-11-06 17:18
Mark, can this bug be closed, then?
msg94987 - (view) Author: Mark Dickinson (mark.dickinson) Date: 2009-11-06 17:29
Well, test_signal is still failing.  So I guess this should stay open 
until we deal with the failure one way or another.  (E.g. skipping the 
test, or marking the failure as expected, or ...)
msg94988 - (view) Author: R. David Murray (r.david.murray) Date: 2009-11-06 17:41
Put in a platform+release specific skip, perhaps.
History
Date User Action Args
2009-11-06 17:42:05r.david.murraysetkeywords: + buildbot
2009-11-06 17:41:20r.david.murraysetpriority: normal
keywords: + easy, - buildbot
messages: + msg94988
2009-11-06 17:29:32mark.dickinsonsetmessages: + msg94987
2009-11-06 17:18:50r.david.murraysetmessages: + msg94984
2009-10-31 11:08:48mark.dickinsonsetnosy: + mark.dickinson
messages: + msg94753
2009-10-30 20:48:37r.david.murraysetnosy: aimacintyre, eric.smith, gpolo, r.david.murray
messages: + msg94724

components: + Tests
type: behavior
stage: needs patch
2009-10-30 20:27:59eric.smithsetnosy: + eric.smith
messages: + msg94722
2009-10-30 20:01:26r.david.murraysetversions: + Python 3.1, Python 2.7, Python 3.2
nosy: + r.david.murray

messages: + msg94721

keywords: + buildbot
2008-09-28 06:46:15aimacintyresetmessages: + msg73956
2008-09-23 13:32:22gpolosetmessages: + msg73638
2008-09-23 13:30:43aimacintyresetmessages: + msg73637
2008-09-23 13:27:01aimacintyresetmessages: + msg73636
2008-09-23 13:03:38gpolosetmessages: + msg73633
2008-09-23 12:47:03aimacintyresetmessages: + msg73632
2008-09-23 12:40:03aimacintyresetmessages: + msg73630
2008-09-23 12:01:46aimacintyresetmessages: + msg73627
2008-09-23 11:52:33aimacintyresetmessages: + msg73626
2008-09-23 11:48:30aimacintyresetmessages: + msg73625
2008-09-15 10:32:12aimacintyresetmessages: + msg73253
2008-09-14 17:58:22gpolosetnosy: + gpolo
messages: + msg73231
2008-09-14 11:42:24aimacintyresetversions: + Python 2.6
2008-09-14 11:41:53aimacintyrecreate