classification
Title: multiprocessing Pool instantiation crashes on 64 bit 2.6.6rc1 under Windows 7
Type: behavior Stage: resolved
Components: Extension Modules, Library (Lib), Windows Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brian.curtin Nosy List: barry, brian.curtin, cipater, jnoller, loewis
Priority: release blocker Keywords: patch

Created on 2010-08-14 03:30 by cipater, last changed 2010-08-14 19:36 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
issue9600.diff brian.curtin, 2010-08-14 15:25
Messages (10)
msg113864 - (view) Author: (cipater) Date: 2010-08-14 03:30
I'm running 2.6.6rc1 64 bit on Windows 7

I get the following error when I try to instantiate a Pool:

>>> import multiprocessing
>>> p = Pool(processes = 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\Python26\lib\multiprocessing\__init__.py", line 227, in Pool
    return Pool(processes, initializer, initargs)
  File "c:\Python26\lib\multiprocessing\pool.py", line 84, in __init__
    self._setup_queues()
  File "c:\Python26\lib\multiprocessing\pool.py", line 130, in _setup_queues
    from .queues import SimpleQueue
  File "c:\Python26\lib\multiprocessing\queues.py", line 22, in <module>
    from multiprocessing.synchronize import Lock, BoundedSemaphore, Semaphore, Condition
  File "c:\Python26\lib\multiprocessing\synchronize.py", line 22, in <module>
    from multiprocessing.forking import assert_spawning, Popen
  File "c:\Python26\lib\multiprocessing\forking.py", line 153, in <module>
    from ._multiprocessing import win32, Connection, PipeConnection
ImportError: No module named _multiprocessing
>>>
msg113865 - (view) Author: (cipater) Date: 2010-08-14 03:39
prior to this I already imported Pool (from multiprocessing import Pool)

I also tried the 64 bit 2.6.5 and got the same error.
msg113873 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-08-14 05:48
I just fixed a similar problem for #9513 but the problem was only ever seen when test_multiprocessing was run through regrtest on 3.x on Windows. Removing the relative import dot on _multiprocessing imports let them succeed.

I've never actually seen this cause a problem on an installed version, though. I'll check out 2.6.6 rc1 tomorrow when I get back to a Windows box to see if I can reproduce this.



Reclassifying from crash to behavior - it's just an ImportError, not actually a crash
msg113875 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-08-14 05:55
I should correct myself. As seen on the 2.6 Win7 buildbot[0], test_multiprocessing is skipped there likely due to the "from ._multiprocessing..." ImportError. I kind of remember this now but didn't want to backport the fix to release26-maint since it was frozen, and this didn't appear to be an actual bug in the installed version. As said before, I'll investigate tomorrow.



[0] http://www.python.org/dev/buildbot/builders/x86%20Windows7%202.6/builds/547/steps/test/logs/stdio
msg113902 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-08-14 14:51
Ok, it turns out this is in fact a regression from 2.6.5. My prior investigation for that 3.x issue must not have been on the 2.6 version I thought it was.

Barry: the fix from #9513 (e.g., r83722) will correct this.
msg113908 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-08-14 15:19
Please upload a patch specifically for 2.6.6 finally so that I can review it (and test it locally).  It *seems* safe enough so if you and I both have a high confidence it won't regress anything, and it can go in before Monday, then I might approve it.

I still can't build on Windows 7 w/VS2010 though unfortunately, but I will locally test Ubuntu (Lucid, Maverick), Debian, and OS X 10.6.
msg113909 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-08-14 15:25
Here's the patch. All four changes are one char each and live within win32-specific blocks.
msg113910 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-08-14 16:33
Looks okay to me on *nix.  You can check this in as long as test -u all passes on Windows.  Please include a NEWS file entry and close this issue when it's committed.  Thanks!
msg113914 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-08-14 17:10
Committed in r84031.

Thanks for reporting this, cipater.
msg113925 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-14 19:36
Notice that this is also a duplicate of #9144, which was fixed for 2.7, but nobody bothered to move the fix to the other branches. This goes all back to #7902, which has now introduced an incompatible change into 2.6 (otherwise, changing multiprocessing shouldn't have been necessary, despite that having been wrong all the time). I'll reopen that as release-critical for review.
History
Date User Action Args
2010-08-14 19:36:46loewissetnosy: + loewis
messages: + msg113925
2010-08-14 17:10:57brian.curtinsetstatus: open -> closed
messages: + msg113914

assignee: brian.curtin
resolution: accepted -> fixed
stage: needs patch -> resolved
2010-08-14 16:33:17barrysetresolution: accepted
messages: + msg113910
2010-08-14 15:25:24brian.curtinsetfiles: + issue9600.diff
keywords: + patch
messages: + msg113909
2010-08-14 15:19:18barrysetmessages: + msg113908
2010-08-14 14:51:49brian.curtinsetpriority: normal -> release blocker

messages: + msg113902
2010-08-14 05:55:50brian.curtinsetmessages: + msg113875
2010-08-14 05:49:00brian.curtinsetnosy: + brian.curtin
messages: + msg113873

components: + Library (Lib), Windows
type: crash -> behavior
stage: needs patch
2010-08-14 03:54:49eric.araujosetnosy: + barry, jnoller
2010-08-14 03:39:53cipatersetmessages: + msg113865
2010-08-14 03:30:31cipatercreate