New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
defaultdict raises SystemError, __missing__ returned NULL in thread #74658
Comments
Working on this PR (vericast/spylon#49), I encountered an unexpected SystemError in Python 3.6 on my Mac and on Travis Linux. Exception in thread Thread-4:
Traceback (most recent call last):
File "/Users/parente/miniconda3/envs/spylon-dev/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/Users/parente/miniconda3/envs/spylon-dev/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/Users/parente/projects/spylon/spylon/spark/progress.py", line 99, in _spark_progress_thread_worker
td = datetime.datetime.now() - start_times[stage_id]
SystemError: <built-in method __missing__ of collections.defaultdict object at 0x106013098> returned NULL without setting an error The exception only occurs in Python 3.6, not in Python 3.5 or 3.4: https://travis-ci.org/maxpoint/spylon/builds/235992988 The defaultdict is both created and used as a local variable in a threading.Thread run() function. It's never accessed outside that context (vericast/spylon@2bd47dc#diff-1ba10d1cd92a35a380442cb586e310e2R179) I didn't see a related issue on the tracker here nor did I see anything in the 3.6 changelog about a behavior change. |
Much simpler example: https://gist.github.com/parente/a4772297459f05e43e12a5820051431b Wrapping the datetime.datetime.now in a lambda avoids the issue. |
Another way to reproduce this problem: >>> import datetime
>>> from collections import defaultdict as dd
>>> d = dd(datetime.datetime.now)
>>> d['foo']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
SystemError: <built-in method __missing__ of collections.defaultdict object at 0x7f072e55b3b8> returned NULL without setting an error This problem doesn't appear on Python 2.7, too. |
Interestingly I could only reproduce the failure on 3.6 but not master. |
In 3.6 _PyStack_UnpackDict() returns just args if there are no keyword arguments. But if args is NULL, the result of _PyStack_UnpackDict() is treated as error. Thank you for the reproducer Louie! |
This bug is a duplicate of bpo-30524. Louie Lu: "This bug is introduce at commit: 37e4ef7, then fixed at commit: 998c209" This is the fix for master. For Python 3.6, the fix is the commit f0ff849. I added wrote unit tests for this bug, but also for many ways to calls functions: commit b757745.
I'm able to reproduce the bug in Python 3.6 before f0ff849, and also confirm that this commit fixes the bug. The good news is that Python 3.6.2rc1 is already available and contains my fix, and 3.6.2 final should be released soon. |
Oh, and thank you Peter Parente for the bug report, and for the script to reproduce the bug. It was usual to validate that the bug is now fixed. |
Glad to hear it. Cheers, Victor! |
Crap, I wanted to write: useful, not usual!? |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: