Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

In multiprocessing.pool, ExceptionWithTraceback should derive from Exception #65179

Closed
myint mannequin opened this issue Mar 19, 2014 · 6 comments
Closed

In multiprocessing.pool, ExceptionWithTraceback should derive from Exception #65179

myint mannequin opened this issue Mar 19, 2014 · 6 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@myint
Copy link
Mannequin

myint mannequin commented Mar 19, 2014

BPO 20980
Nosy @bitdancer
Files
  • exception_with_traceback.diff
  • thread_pool_exception_test.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2014-03-23.18:42:37.862>
created_at = <Date 2014-03-19.14:53:54.411>
labels = ['type-bug', 'library']
title = 'In multiprocessing.pool, ExceptionWithTraceback should derive from Exception'
updated_at = <Date 2014-03-23.18:42:37.861>
user = 'https://bugs.python.org/myint'

    bugs.python.org fields:

    activity = <Date 2014-03-23.18:42:37.861>
actor = 'sbt'
assignee = 'sbt'
closed = True
closed_date = <Date 2014-03-23.18:42:37.862>
closer = 'sbt'
components = ['Library (Lib)']
creation = <Date 2014-03-19.14:53:54.411>
creator = 'myint'
dependencies = []
files = ['34518', '34520']
hgrepos = []
issue_num = 20980
keywords = ['patch']
message_count = 6.0
messages = ['214101', '214102', '214106', '214247', '214273', '214571']
nosy_count = 4.0
nosy_names = ['r.david.murray', 'python-dev', 'sbt', 'myint']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20980'
versions = ['Python 3.4', 'Python 3.5']
    @myint
    Copy link
    Mannequin Author

    myint mannequin commented Mar 19, 2014

    In multiprocessing.pool, ExceptionWithTraceback is not derived from Exception. Thus when it is raised, we get the exception, "TypeError: exceptions must derive from BaseException". Attached is a patch that fixes this.

    See below example traceback that prompted me to notice this:

    Traceback (most recent call last):
  File "/Users/myint/Library/Python/3.4/bin/yolk", line 9, in <module>
    load_entry_point('yolk3k==0.7.3', 'console_scripts', 'yolk')()
  File "/Users/myint/Library/Python/3.4/lib/python/site-packages/yolk/cli.py", line 1123, in main
    my_yolk.run()
  File "/Users/myint/Library/Python/3.4/lib/python/site-packages/yolk/cli.py", line 208, in run
    return getattr(self, action)()
  File "/Users/myint/Library/Python/3.4/lib/python/site-packages/yolk/cli.py", line 240, in show_updates
    for (project_name, version, newest) in _updates(pkg_list, self.pypi):
  File "/Users/myint/Library/Python/3.4/lib/python/site-packages/yolk/cli.py", line 1104, in _updates
    names):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 255, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 594, in get
    raise self._value
TypeError: exceptions must derive from BaseException

    @myint myint mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 19, 2014
    @bitdancer
    Copy link
    Member

    We must have a missing test case, then. One should be added.

    @myint
    Copy link
    Mannequin Author

    myint mannequin commented Mar 19, 2014

    I didn't quite understand the multiprocessing test cases, but attached is a standalone test case that reproduces the problem. The problem only shows up when using ThreadPool.

    $ python thread_pool_exception_test.py
Traceback (most recent call last):
  File "thread_pool_exception_test.py", line 7, in <module>
    pool.map(exception, [1])
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 255, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 594, in get
    raise self._value
TypeError: exceptions must derive from BaseException

    @myint
    Copy link
    Mannequin Author

    myint mannequin commented Mar 20, 2014

    This bug was introduced in bpo-13831.

    @sbt
    Copy link
    Mannequin

    sbt mannequin commented Mar 20, 2014

    We should only wrap the exception with ExceptionWithTraceback in the process case where it will be pickled and then unpickled.

    @sbt sbt mannequin self-assigned this Mar 20, 2014
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Mar 23, 2014

    New changeset df6a6951b2c9 by Richard Oudkerk in branch '3.4':
    Issue bpo-20980: Stop wrapping exception when using ThreadPool.
    http://hg.python.org/cpython/rev/df6a6951b2c9

    @sbt sbt mannequin closed this as completed Mar 23, 2014
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @bitdancer