test_3_join_in_forked_from_thread() of test_threading hangs 1 hour on "x86 Ubuntu Shared 3.x" #56079
Comments
|
test_3_join_in_forked_from_thread() of test_threading failed on "x86 Ubuntu Shared 3.x" buildbot: Code of the test: def _run_and_join(self, script):
script = """if 1:
import sys, os, time, threading
# a thread, which waits for the main program to terminate
def joiningfunc(mainthread):
mainthread.join()
print('end of thread')
# stdout is fully buffered because not a tty, we have to flush
# before exit.
sys.stdout.flush()
\n""" + script
p = subprocess.Popen([sys.executable, "-c", script], stdout=subprocess.PIPE)
rc = p.wait() <~~~ HANG HERE ~~~~
data = p.stdout.read().decode().replace('\r', '')
p.stdout.close()
self.assertEqual(data, "end of main\nend of thread\n")
self.assertFalse(rc == 2, "interpreter was blocked")
self.assertTrue(rc == 0, "Unexpected error") t = threading.Thread(target=joiningfunc,
args=(main_thread,))
print('end of main')
t.start()
t.join() # Should not block: main_thread is already stopped
w = threading.Thread(target=worker)
w.start()
"""
self._run_and_join(script) |
|
test_2_join_in_forked_process fails on FreeBSD 6.4 buildbot. """ Traceback (most recent call last):
File "/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_threading.py", line 464, in test_2_join_in_forked_process
self._run_and_join(script)
File "/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_threading.py", line 436, in _run_and_join
self.assertEqual(data, "end of main\nend of thread\n")
AssertionError: '' != 'end of main\nend of thread\n'
+ end of main
+ end of thread
"""I think it's the same problem as issue bpo-12316: in the child process, even calling pthread_create can segfault/abort on FreeBSD6 (async-safe blahblah...). As for test_3_join_in_forked_from_thread, well, it could be more or less the same problem. We're really doing something prohibited by POSIX, so things might break in unexpected ways. For example, calling pthread_condition_destroy from the child process can deadlock (see http://bugs.python.org/issue6721#msg136047). Victor: to debug this kind of problem, it would be great if faulthandler could also dump tracebacks of children processes. Do you mind if I create a new issue? |
Please open a new issue. |
|
+ @unittest.skipIf(sys.platform in ('freebsd4', 'freebsd5', 'freebsd6', This skip gives very few information, and it is duplicated for each function. I would prefer a constant of the "broken OSes" with your following comment attached to the constant: + # Between fork() and exec(), only async-safe functions are allowed (issues Or split the test case into two testcases: one using fork and skipped on broken platforms, one not using fork? ---
If the creation of a thread after a fork is reliable on some systems, we should not deny the creation of new threads after a fork. You can replace "creation of new threads" by any other non async-safe function in my previous sentence. Therefore I agree that the good answer to this issue is to skip the test on "broken systems" (or should we call them "POSIX compliant systems?" :-)). |
Ok, I'll try to write something along those lines.
Well, the problem is that it is not reliable on any platform, but happens to work "most of the time" on some platforms ;-) |
|
Patch attached. |
|
Your patch is linux3 compliant, go ahead! |
|
New changeset 0ed5e6ff10f8 by Victor Stinner in branch '3.2': New changeset f43dee86fffd by Victor Stinner in branch 'default': |
|
New changeset ff36b8cadfd6 by Victor Stinner in branch '2.7': |
|
The initial problem was test_3_join_in_forked_from_thread() and the hangs does still happen: [324/356] test_threading (neologix's patch doesn't change anything for x86 Ubuntu Shared 3.x buildbot, which is a Linux, not a FreeBSD) I don't know why it only hangs on this Linux buildbot. It's maybe an old Linux kernel, an old GNU libc version, or something like that? |
Yes, the patch was there to fix test_2_join_in_forked_from_thread.
This means that the subprocess hangs, but without a backtrace of the So I suggest to try to come up with a solution to bpo-12413, which should |
The subprocess hang still occurs something, it just happened: [110/363] test_threading |
|
To debug this, we should probably make use of faulthandler (but not |
|
Here's a patch to help nail this down. |
|
Victor, could you try the patch attached? |
|
New changeset 775319cebaa3 by Charles-François Natali in branch '2.7': New changeset de962ec0faaa by Charles-François Natali in branch '3.2': New changeset cec0d77d01c4 by Charles-François Natali in branch 'default': |
|
Should be fixed now. |
|
New changeset cd54b48946ca by Stefan Krah in branch '3.3': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: