This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Pickle not deserializing an aiohttp ClientConnectorError exception as expected
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: davidparks21, serhiy.storchaka, xtreak
Priority: normal Keywords:

Created on 2019-09-22 18:02 by davidparks21, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg352985 - (view) Author: David Parks (davidparks21) Date: 2019-09-22 18:02
Below is a minimum reproducible test case of what appears to be an issue in pickle. 

Before pickling the exception ClientConnectionError, from aiohttp, the property ClientConnectionError._os_error is a PermissionError object (a subclass of OSError). After deserialization ClientConnectionError._os_error appears to be a string based on the exception that is produced. 

I'm not familiar enough with pickle to make much headway here.

```
import aiohttp
import pickle


connection_key = aiohttp.client_reqrep.ConnectionKey
ose = OSError(1, 'unittest')
cce = aiohttp.client_exceptions.ClientConnectorError(connection_key, ose)
cce_pickled = pickle.dumps(cce)
pickle.loads(cce_pickled)
```

```
Traceback (most recent call last):
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1758, in <module>
    main()
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1752, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1147, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/project_neural_mouse/src/ARCHIVE/scratch/scratch_13.py", line 11, in <module>
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client_exceptions.py", line 133, in __init__
    super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
```
msg352997 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-23 09:22
aiohttp is not the part of the standard Python library. Please use an appropriate bug tracker for reporting this issue.
msg353038 - (view) Author: David Parks (davidparks21) Date: 2019-09-23 18:45
I may be wrong here, but the issue appears to be a problem in pickle, which is why I brought it over here. From the looks of the very simple code in the Exception I can't see that there's any way that this exception is possible unless pickle itself has a bug. Pickle appears to be reconstructing the object with a string in a place where an Object was before the serialization process.

The issue is currently reported under aiohttp here: https://github.com/aio-libs/aiohttp/issues/4077

It's also posted (originally) on stack overflow here: https://stackoverflow.com/questions/58019939/attributeerror-str-object-has-no-attribute-errno
msg353095 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-09-24 15:30
A simplified example to reproduce the issue as below. It seems that the OSError instance passed to the constructor is pickled such that during unpickling the string part of OSError 'unittest' is passed to the constructor instead of the original OSError object.


import pickle

ose = OSError(1, 'unittest')

class SubOSError(OSError):

    def __init__(self, foo, os_error):
        super().__init__(os_error.errno, os_error.strerror)

cce = SubOSError(1, ose)
cce_pickled = pickle.dumps(cce)
pickle.loads(cce_pickled)


./python.exe ../backups/bpo38254.py
Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/../backups/bpo38254.py", line 12, in <module>
    pickle.loads(cce_pickled)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/../backups/bpo38254.py", line 8, in __init__
    super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
msg353101 - (view) Author: David Parks (davidparks21) Date: 2019-09-24 16:59
An answer came in over the stack overflow question. 

https://stackoverflow.com/questions/58019939/attributeerror-str-object-has-no-attribute-errno/58084380#58084380

The issue is that OSError implements a custom __reduce__ function which needs to be override by the subclass when the arguments to the subclass are different than the parent. 

So no bug, just pickle subtlety. Passing it back to the aiohttp repo for a fix there.

https://github.com/aio-libs/aiohttp/issues/4077
msg353102 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-09-24 17:04
Thanks for the clarification and the answer link. I was also just debugging along the similar lines around how OSError.__reduce__ treats the arguments something like. Custom __reduce__ for aiohttp works as explained in the SO answer.


class SubOSError(OSError):

    def __init__(self, foo, os_error, /):
        self.foo = foo
        self.os_error = os_error
        super().__init__(os_error.errno, os_error.strerror)

    def __reduce__(self): # Custom __reduce__
        return (self.__class__, (self.foo, self.os_error ))

    def __reduce__(self): # OSError __reduce__
        return (self.__class__, (self.os_error.errno, self.os_error.strerror ))
History
Date User Action Args
2022-04-11 14:59:20adminsetgithub: 82435
2019-09-24 17:04:59xtreaksettype: crash -> behavior
messages: + msg353102
2019-09-24 16:59:24davidparks21setstatus: open -> closed
resolution: not a bug
messages: + msg353101
2019-09-24 15:30:18xtreaksetnosy: + xtreak
messages: + msg353095
2019-09-23 18:45:52davidparks21setstatus: closed -> open
resolution: third party -> (no value)
messages: + msg353038
2019-09-23 09:22:27serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg352997

resolution: third party
stage: resolved
2019-09-22 18:06:28davidparks21settitle: Pickle not deserializing an OSError exception as expected -> Pickle not deserializing an aiohttp ClientConnectorError exception as expected
2019-09-22 18:02:44davidparks21create