classification
Title: multiprocessing.Pool windows/linux behaviour difference
Type: Stage:
Components: Library (Lib) Versions: Python 3.0
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, mark, pitrou
Priority: normal Keywords:

Created on 2008-08-19 06:50 by mark, last changed 2008-08-19 09:09 by mark. This issue is now closed.

Files
File name Uploaded Description Edit
mtest.py mark, 2008-08-19 06:50 Simple exercise of multiprocessing.Pool
Messages (4)
msg71408 - (view) Author: Mark Summerfield (mark) * Date: 2008-08-19 06:50
When the attached program is run on Linux it runs "instantly" and
outputs one line, e.g.:

$ python3 mtest.py
100 files, 1702627142 bytes

(The number of bytes will vary depending on the system.)

When run on Windows XP there is no output at all; many processes seem to
be created but nothing seems to actually get done.

In both cases I'm using Py30b2.
msg71409 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-19 08:49
There is a problem in your script.

On Windows, the remote processes have to import the mtest module, in
order to execute the get_size() function.
This in turn calls the main() function, which creates a new pool of
processes, etc... exponential explosion.
Linux forks and does not have this problem.

The solution of course is

if __name__ == '__main__':
    main()
msg71410 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-19 09:06
For what it's worth, this is documented in
http://docs.python.org/dev/library/multiprocessing.html#windows
msg71411 - (view) Author: Mark Summerfield (mark) * Date: 2008-08-19 09:09
On 2008-08-19, Antoine Pitrou wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> For what it's worth, this is documented in
> http://docs.python.org/dev/library/multiprocessing.html#windows

Ah yes, sorry I missed that.
History
Date User Action Args
2008-08-19 09:09:58marksetmessages: + msg71411
2008-08-19 09:06:09pitrousetnosy: + pitrou
messages: + msg71410
2008-08-19 08:49:37amaury.forgeotdarcsetstatus: open -> closed
resolution: not a bug
messages: + msg71409
nosy: + amaury.forgeotdarc
2008-08-19 06:50:56markcreate