This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Wicken
Recipients Wicken
Date 2021-01-15.12:05:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1610712338.22.0.647900497597.issue42935@roundup.psfhosted.org>
In-reply-to
Content
Originally found as an issue in Lib/shelve.py; if we attempt to pickle a builtin as the program is exiting then Modules/_pickle.c will fail at the point of the PyImport_Import in save_global. In CPython3.8 this causes a segfault, in CPython3.9 a PicklingError is raised.

This is especially problematic in shelve.py as object pickling is attempted by the __del__ method's call stack when writeback=True. Therefore if the program exits before an explicit sync is called; in 3.8 the data will not be written to disk and a segfault occurs; in 3.9 the data is written to disk, but with an uncaught exception being raised.


Exception demonstrated via shelve on 3.9.1 on MacOS with Clang:

Python 3.9.1 (default, Dec 10 2020, 11:11:14)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
Exception ignored in: <function Shelf.__del__ at 0x10ed67790>
Traceback (most recent call last):
  File ".../3.9/lib/python3.9/shelve.py", line 162, in __del__
  File ".../3.9/lib/python3.9/shelve.py", line 144, in close
  File ".../3.9/lib/python3.9/shelve.py", line 168, in sync
  File ".../3.9/lib/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module 'builtins' failed


Segfault demonstrated via shelve on 3.8.5 on MacOS with Clang (different system from above):

Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
[1]    10040 segmentation fault  python3.8


Exception demonstrated via shelve on 3.9.1 on RHEL with GCC:

Python 3.9.1 (default, Dec  8 2020, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open("thing", writeback=True)
>>> s['a'] = Exception
>>> 
Exception ignored in: <function Shelf.__del__ at 0x7f3f30b7c820>
Traceback (most recent call last):
  File "/usr/lib64/python3.9/shelve.py", line 162, in __del__
  File "/usr/lib64/python3.9/shelve.py", line 144, in close
  File "/usr/lib64/python3.9/shelve.py", line 168, in sync
  File "/usr/lib64/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module 'builtins' failed


Code example to reproduce using Pickle in the class __del__, demonstrated on a RHEL system:

Python 3.9.1 (default, Dec  8 2020, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pickle import DEFAULT_PROTOCOL, Pickler
>>> from io import BytesIO
>>> class T:
...   def __del__(self):
...     f = BytesIO()
...     p = Pickler(f, DEFAULT_PROTOCOL)
...     p.dump(sum)
... 
>>> t = T()
>>> exit()
Exception ignored in: <function T.__del__ at 0x7f5f04d9ef70>
Traceback (most recent call last):
  File "<stdin>", line 5, in __del__
_pickle.PicklingError: Can't pickle <built-in function sum>: import of module 'builtins' failed


Have not tested on 3.6, 3.7 or 3.10.
History
Date User Action Args
2021-01-15 12:05:38Wickensetrecipients: + Wicken
2021-01-15 12:05:38Wickensetmessageid: <1610712338.22.0.647900497597.issue42935@roundup.psfhosted.org>
2021-01-15 12:05:38Wickenlinkissue42935 messages
2021-01-15 12:05:37Wickencreate