This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Module variable overridden in child processes with multiprocessing
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: TarkaSteve, amaury.forgeotdarc, benjamin.peterson, pitrou
Priority: normal Keywords:

Created on 2008-09-06 04:53 by TarkaSteve, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (5)
msg72662 - (view) Author: Steve Smith (TarkaSteve) Date: 2008-09-06 04:53
The process variable 'p' is leaking into sub-processes when using the
multiprocessing modules.  The following code demonstrates the problem:

    import sys
    from multiprocessing import Process

    p = 'Correct'

    def test():
        print "Got 'p' of", p

    if __name__ == '__main__':
        if len(sys.argv) == 2 and sys.argv[1] == '-m':
            p = Process(target=test)
            p.start()
            p.join()
        else:
            test()

Running this in SP and MP mode shows the leakage:

    ssmith$ /opt/python-svn/bin/python mpbug.py 
    Got 'p' of Correct
    ssmith$ /opt/python-svn/bin/python mpbug.py -m
    Got 'p' of <Process(Process-1, started)>

This occurs in both 2.6b3 and trunk.
msg72663 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-09-06 10:43
What do you mean, "overriden"? The behaviour looks totally normal to me.
You change the value of "p" before the subprocess is started, so the
subprocess inherits the new value, not the old one.
msg72664 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-09-06 12:28
I agree; this is the expected behavior. Perhaps you don't understand how
globals work in Python?
msg72665 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-09-06 12:46
Well, I get the OP's expected result on windows:

C:\dev\python\trunk>PCbuild\python_d t.py
Got 'p' of Correct
C:\dev\python\trunk>PCbuild\python_d t.py -m
Got 'p' of Correct

This is easy to explain: on Unix, the forked process has a copy of the
memory and reads the last value in the module. 
But on Windows, the freshly spawned process imports the module, and get
the initial value (since it does not enter the "__main__" block).

This is documented:
http://docs.python.org/dev/library/multiprocessing.html#windows , under
"Global Variables".

By the way, the Windows way may have some advantages for some uses.
Could the same method (start a new interpreter, import modules, copy
needed objects), be made available on Unix?
msg72670 - (view) Author: Steve Smith (TarkaSteve) Date: 2008-09-06 14:13
Ugh, sorry.  Stupidity on the reporter's part.
History
Date User Action Args
2022-04-11 14:56:38adminsetgithub: 48042
2008-09-06 14:13:02TarkaStevesetmessages: + msg72670
2008-09-06 12:46:37amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg72665
2008-09-06 12:28:45benjamin.petersonsetstatus: open -> closed
resolution: not a bug
messages: + msg72664
nosy: + benjamin.peterson
2008-09-06 10:43:06pitrousetnosy: + pitrou
messages: + msg72663
2008-09-06 04:53:05TarkaStevecreate