Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datetime object fails to restore from reduction #72938

Closed
jaraco opened this issue Nov 20, 2016 · 13 comments
Closed

datetime object fails to restore from reduction #72938

jaraco opened this issue Nov 20, 2016 · 13 comments
Assignees
Labels
3.7 (EOL) end of life extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@jaraco
Copy link
Member

jaraco commented Nov 20, 2016

BPO 28752
Nosy @tim-one, @jaraco, @abalkin, @vstinner, @avassalotti, @ned-deily, @serhiy-storchaka
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • datetime-reduce.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/abalkin'
    closed_at = <Date 2016-11-21.22:35:22.500>
    created_at = <Date 2016-11-20.15:55:23.401>
    labels = ['extension-modules', 'type-bug', '3.7']
    title = 'datetime object fails to restore from reduction'
    updated_at = <Date 2017-03-31.16:36:16.587>
    user = 'https://github.com/jaraco'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:16.587>
    actor = 'dstufft'
    assignee = 'belopolsky'
    closed = True
    closed_date = <Date 2016-11-21.22:35:22.500>
    closer = 'belopolsky'
    components = ['Extension Modules']
    creation = <Date 2016-11-20.15:55:23.401>
    creator = 'jaraco'
    dependencies = []
    files = ['45564']
    hgrepos = []
    issue_num = 28752
    keywords = ['patch']
    message_count = 13.0
    messages = ['281278', '281279', '281280', '281284', '281287', '281288', '281292', '281388', '281395', '281396', '281397', '281399', '281521']
    nosy_count = 8.0
    nosy_names = ['tim.peters', 'jaraco', 'belopolsky', 'vstinner', 'alexandre.vassalotti', 'ned.deily', 'python-dev', 'serhiy.storchaka']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28752'
    versions = ['Python 3.6', 'Python 3.7']

    @jaraco
    Copy link
    Member Author

    jaraco commented Nov 20, 2016

    On Python 3.5, the datetime would reduce and restore cleanly.

    $ python3.5 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)"

    With Python 3.6.0b3, it now fails with a TypeError.

    $ python3.6 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)"
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: an integer is required (got type bytes)

    @jaraco jaraco added the 3.7 (EOL) end of life label Nov 20, 2016
    @jaraco
    Copy link
    Member Author

    jaraco commented Nov 20, 2016

    Pickle still works, so pickle must be relying on a different protocol for serialization.

    $ python
    Python 3.6.0b3 (v3.6.0b3:8345e066c0ed, Oct 31 2016, 18:05:23) 
    [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pickle
    >>> import datetime
    >>> pickle.loads(pickle.dumps(datetime.datetime.now()))
    datetime.datetime(2016, 11, 20, 10, 56, 43, 264436)

    @serhiy-storchaka
    Copy link
    Member

    Now pickling of the datetime.datetime objects is implemented with __reduce_ex__. __reduce__ is not defined in datetime.datetime and is inherited from datetime.date.

    >>> datetime.datetime.__reduce_ex__
    <method '__reduce_ex__' of 'datetime.datetime' objects>
    >>> datetime.datetime.__reduce__
    <method '__reduce__' of 'datetime.date' objects>

    __reduce_ex__ has higher priority and is called instead of __reduce__. It is weird that __reduce_ex__ and __reduce__ are not consistent. __reduce__ should be defined as

        def __reduce__(self):
            return self.__reduce_ex__(2)

    or just

        __reduce__ = object.__reduce__

    Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

    @serhiy-storchaka serhiy-storchaka added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Nov 20, 2016
    @serhiy-storchaka
    Copy link
    Member

    Proposed patch restores the __reduce__() methods and makes Python and C implementations more consistent.

    @abalkin
    Copy link
    Member

    abalkin commented Nov 20, 2016

    On Nov 20, 2016, at 11:10 AM, Serhiy Storchaka <report@bugs.python.org> wrote:

    Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

    I would prefer this solution.

    @serhiy-storchaka
    Copy link
    Member

    Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

    Sorry, I was wrong. This would wouldn't work with C implementation. And explicitly setting __reduce__ = object.__reduce__ doesn't work. The only way is to define both __reduce_ex__ and __reduce__ for time and datewtime.

    @abalkin
    Copy link
    Member

    abalkin commented Nov 20, 2016

    On Nov 20, 2016, at 12:34 PM, Serhiy Storchaka <report@bugs.python.org> wrote:

    The only way is to define both __reduce_ex__ and __reduce__ for time and datewtime.

    OK. I'll review your patch and get it committed shortly.

    @abalkin abalkin self-assigned this Nov 21, 2016
    @abalkin
    Copy link
    Member

    abalkin commented Nov 21, 2016

    The patch LGTM. I'll commit it tonight unless Serhiy beats me to it.

    @ned-deily
    Copy link
    Member

    If you can push this in the next hour or two, it can still make b4.

    @abalkin
    Copy link
    Member

    abalkin commented Nov 21, 2016

    If you can push this in the next hour or two ...

    I'll do it now.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 21, 2016

    New changeset 0a2a0061e425 by Serhiy Storchaka in branch '3.6':
    Issue bpo-28752: Restored the __reduce__() methods of datetime objects.
    https://hg.python.org/cpython/rev/0a2a0061e425

    New changeset 23140bd66d86 by Serhiy Storchaka in branch 'default':
    Issue bpo-28752: Restored the __reduce__() methods of datetime objects.
    https://hg.python.org/cpython/rev/23140bd66d86

    @abalkin
    Copy link
    Member

    abalkin commented Nov 21, 2016

    It looks like Serhiy has already committed it. Thanks!

    @abalkin abalkin closed this as completed Nov 21, 2016
    @jaraco
    Copy link
    Member Author

    jaraco commented Nov 22, 2016

    Thanks all! So pleased to see this fixed.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants