Author Olivier.Grisel
Recipients Olivier.Grisel, brett.cannon, eric.snow, ncoghlan, pitrou, python-dev, sbt
Date 2013-12-13.19:18:24
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1386962304.94.0.823687753775.issue19946@psf.upfronthosting.co.za>
In-reply-to
Content
I can wait (or monkey-patch the stuff I need as a temporary workaround in my code). My worry is that Python 3.4 will introduce a new feature that is very crash-prone.

Take this simple program that uses the newly introduced `get_context` function (the same problem happens with `set_start_method`):

filename: mytool
"""
#!/usr/bin/env python
from multiprocessing import freeze_support, get_context


def compute_stuff(i):
    # in real life you could use a lib that uses threads
    # like cuda and that would crash with the default 'fork'
    # mode under POSIX
    return i ** 2


if __name__ == "__main__":
     freeze_support()
     ctx = get_context('spawn')
     ctx.Pool(4).map(compute_stuff, range(8))

"""

If you chmod +x this file and run it with ./mytool, the user will get an infinitely running process that keeps displaying on stderr:

"""
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 96, in spawn_main
    exitcode = _main(fd)
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 105, in _main
    prepare(preparation_data)
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 210, in prepare
    import_main_path(data['main_path'])
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 256, in import_main_path
    raise ImportError(name=main_name)
ImportError
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 96, in spawn_main
    exitcode = _main(fd)
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 105, in _main
    prepare(preparation_data)
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 210, in prepare
    import_main_path(data['main_path'])
  File "/opt/Python-HEAD/lib/python3.4/multiprocessing/spawn.py", line 256, in import_main_path
    raise ImportError(name=main_name)
ImportError
...
"""

until the user kills the process. Is there really nothing we can do to avoid releasing Python 3.4 with this bug?
History
Date User Action Args
2013-12-13 19:18:24Olivier.Griselsetrecipients: + Olivier.Grisel, brett.cannon, ncoghlan, pitrou, python-dev, sbt, eric.snow
2013-12-13 19:18:24Olivier.Griselsetmessageid: <1386962304.94.0.823687753775.issue19946@psf.upfronthosting.co.za>
2013-12-13 19:18:24Olivier.Grisellinkissue19946 messages
2013-12-13 19:18:24Olivier.Griselcreate