classification
Title: test_concurrent_futures fails on FreeBSD
Type: Stage:
Components: Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, bquinlan, loewis, vstinner
Priority: normal Keywords: patch

Created on 2010-12-30 20:14 by loewis, last changed 2014-06-26 22:56 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
fbsd.diff loewis, 2010-12-30 20:14 review
Messages (9)
msg124933 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-12-30 20:14
This is similar to #10348, but has a different scope; the attached patch disables the ProcessPoolExecutor if the system has too few POSIX semaphores.

To keep support for the ThreadPoolExecutor, I had the test cases stop using multiprocessing.Event in the threaded test cases. Unfortunately, this had two side effect that I think indicate a bug elsewhere: 

1. ThreadPoolWaitTests.test_all_completed_some_already_completed hangs
2. (sometimes) ThreadPoolWaitTests.test_first_exception fails:

    self.assertEqual(set([future1, future2]), finished)
AssertionError: Items in the first set but not the second:
<Future at 0x1851ad0 state=running>

I haven't been able to determine yet why it hangs. If the hanging test is disabled, the tests pass on both Linux and FreeBSD 7.3.
msg125080 - (view) Author: Brian Quinlan (bquinlan) (Python committer) Date: 2011-01-02 19:49
Martin,

Could you commit this patch if you think that it is the right thing? I'm going to be restructuring the tests and don't want you to get caught in merge hell.

Cheers,
Brian
msg125136 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-02 23:54
Is this really the time for restructuring tests? I think such activity should wait until after the 3.2 release.
msg125140 - (view) Author: Brian Quinlan (bquinlan) (Python committer) Date: 2011-01-03 00:03
The tests are currently flaky on Windows and I'd like to fix that before the release. But I don't have the bandwidth to debug the Call() abstraction used in the tests before 3.2 RC1.

You can see the test change here:
http://svn.python.org/view/python/branches/py3k-futures-on-windows/Lib/test/test_concurrent_futures.py?r1=87484&r2=87652

So far it is passing on all the platforms that I've tested it on (Mac OS X, Ubuntu and Windows 7).
msg125141 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-03 00:08
Thanks for the review, committed as r87665. Fixing flaky tests is fine, of course.
msg125142 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-03 00:21
Follow-up: I have disabled test_all_completed_some_already_completed in r87667, as I couldn't figure out why it hangs.
msg125250 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-01-03 22:39
test_multiprocessing is still failing with "OSError: [Errno 23] Too many open files in system" on "x86 FreeBSD 7.2 3.x":

http://www.python.org/dev/buildbot/builders/x86%20FreeBSD%207.2%203.x/builds/1396/steps/test/logs/stdio

I don't know the maximum number of POSIX semaphores on this buildbot.
msg125294 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-04 05:57
Interesting. I wonder whether a reboot of the system would help (as it may have old semaphores hanging around).
msg221656 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-26 22:55
Is it safe to assume that this test problem was resolved long ago?
History
Date User Action Args
2014-06-26 22:56:57vstinnersetstatus: open -> closed
resolution: fixed
2014-06-26 22:55:53BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221656
2011-01-04 05:57:16loewissetnosy: loewis, bquinlan, vstinner
messages: + msg125294
2011-01-03 22:39:45vstinnersetnosy: + vstinner
messages: + msg125250
2011-01-03 00:21:55loewissetmessages: + msg125142
2011-01-03 00:08:57loewissetmessages: + msg125141
2011-01-03 00:03:54bquinlansetmessages: + msg125140
2011-01-02 23:54:27loewissetmessages: + msg125136
2011-01-02 19:49:00bquinlansetmessages: + msg125080
2010-12-30 20:14:04loewiscreate