I'm trying to parallelize some scientific computing jobs using multiprocessing.Pool.  I've also tried rolling my own Pool equivalent using Queues.  In trying to return very large result objects from Pool.map()/imap() or via Queue.put(), I've noticed that multiprocessing seems to hang on the receiving end.  On Cygwin 1.7.1/Python 2.5.2 it hangs with no CPU activity.  On Centos 5.2/Python 2.6.2 it hangs with 100% CPU.  cPickle is perfectly capable of pickling these objects, although they may be 100's of MB, so I think it's the communication.  There's also some asymmetry in the error whether it's the parent or child putting the large object.  The put does appear to succeed;  it's the get() on the other end that hangs forever.

Example code:
-----
from multiprocessing import *

def child(task_q, result_q):
    while True:
        print "  Getting task..."
        task = task_q.get()
        print "  Got task", task[:10]
        task = task * 100000000
        print "  Putting result", task[:10]
        result_q.put(task)
        print "  Done putting result", task[:10]
        task_q.task_done()

def parent():
    task_q = JoinableQueue()
    result_q = JoinableQueue()
    worker = Process(target=child, args=(task_q,result_q))
    worker.daemon = True
    worker.start()
    #tasks = ["foo", "bar", "ABC" * 100000000, "baz"]
    tasks = ["foo", "bar", "ABC", "baz"]
    for task in tasks:
        print "Putting task", task[:10], "..."
        task_q.put(task)
        print "Done putting task", task[:10]
    task_q.join()
    for task in tasks:
        print "Getting result..."
        print "Got result", result_q.get()[:10]

if __name__ == '__main__':
    parent()
-----

If run as is, I get
Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/queues.py", line 242, in _feed
    send(obj)
MemoryError: out of memory
(*** hangs, I hit ^C ***)
Got result
Traceback (most recent call last):
Process Process-1:
Traceback (most recent call last):
  File "cygwin_multiprocessing_queue.py", line 32, in <module>
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/process.py", line 237, in _bootstrap
    parent()
  File "cygwin_multiprocessing_queue.py", line 29, in parent
    print "Got result", result_q.get()[:10]
    self.run()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/process.py", line 93, in run
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/queues.py", line 91, in get
    self._target(*self._args, **self._kwargs)
  File "cygwin_multiprocessing_queue.py", line 6, in child
    res = self._recv()
KeyboardInterrupt
    task = task_q.get()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/queues.py", line 91, in get
    res = self._recv()
KeyboardInterrupt


If instead I comment out the multiplication in child() and uncomment the large task in parent(), then I get
  Getting task...
Putting task foo ...
Done putting task foo
Putting task bar ...
  Got task foo
  Putting result foo
Done putting task bar
Putting task ABCABCABCA ...
Done putting task ABCABCABCA
Putting task baz ...
  Done putting result foo
  Getting task...
  Got task bar
  Putting result bar
  Done putting result bar
  Getting task...
Done putting task baz
(*** hangs, I hit ^C ***)
Traceback (most recent call last):
  File "cygwin_multiprocessing_queue.py", line 32, in <module>
    parent()
  File "cygwin_multiprocessing_queue.py", line 26, in parent
    task_q.join()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/queues.py", line 303, in join
    self._cond.wait()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.2.1-py2.5-cygwin-1.7.1-i686.egg/multiprocessing/synchronize.py", line 212, in wait
    self._wait_semaphore.acquire(True, timeout)
KeyboardInterrupt