This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: In Multiprocessing Queue() doesn't work with list : __.put(list) != __.get()
Type: behavior Stage: resolved
Components: Versions: Python 3.3
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: AlainCALMET, amaury.forgeotdarc
Priority: normal Keywords:

Created on 2014-10-10 11:53 by AlainCALMET, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
BisesAmouRmulti-debug.py AlainCALMET, 2014-10-10 11:53 Debug file show that __.put(list) and __.get()
Messages (2)
msg228993 - (view) Author: CALMET (AlainCALMET) Date: 2014-10-10 11:53
Hello,
I have to find the right order of a list, for example
try = [0,1,2,3,4,5,6,7,8,9]
try = [0,1,2,3,4,5,6,7,9,8]
try = [0,1,2,3,4,5,6,8,7,9]
and so on... in this example there are factorial(10) = 3,6 million possible order to test.
The results with be all the list  'try'  that match with the test.

To go faster
- I use mutiproceesiing
- I define   rightorder = Queue() 
- I define a function test   with a line  rightorder.put(try)
- I wait for all the processes are finished  p.join()
- Then I read all the results in the main programm   rightorder.get()

The problem is that on exactly the same example
- the rightorder.put() found by the process are always good
- but the rightorder.get() in the main programm can be different (randomly).

I first believed that when 2 processes call the function  rightorder.put(try)  at the same time, the lists ares mixed together in the Queue().

But even when I force cpu_count() to 1 (mono process) I still have the problem.

To solve the problem I converted the list try=[2,3,0,4,2,9,7,8,1,5]
in a string try2="2304297815"
and since now  rightorder.put(try)  and  rightorder.get()  always give the same results.

So I suppose that in multiprocessing with a Queue() rightorder.put(try)  works if try is a number or a string
but doesn't works if try is a list.
msg231123 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2014-11-13 16:20
"solution.put(c)" takes a *reference* to the list c.
And then the test() function continues running!

By the time the list is serialized and sent back to the main process (in another thread), the test() function has already changed it...

As you noticed, you can use immutable objects instead (string, int), or make a copy of the list: solution.put(c.copy())
History
Date User Action Args
2022-04-11 14:58:08adminsetgithub: 66790
2014-11-13 16:20:20amaury.forgeotdarcsetstatus: open -> closed

nosy: + amaury.forgeotdarc
messages: + msg231123

resolution: not a bug
stage: resolved
2014-10-10 11:53:13AlainCALMETcreate