Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datetime.time issue with pickling in PyPy #86132

Closed
scaramallion mannequin opened this issue Oct 7, 2020 · 5 comments
Closed

datetime.time issue with pickling in PyPy #86132

scaramallion mannequin opened this issue Oct 7, 2020 · 5 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@scaramallion
Copy link
Mannequin

scaramallion mannequin commented Oct 7, 2020

BPO 41966
Nosy @abalkin, @serhiy-storchaka, @pganssle, @miss-islington, @scaramallion
PRs
  • bpo-41966: Fix pickling pure datetime.time subclasses #22731
  • [3.9] bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) #22747
  • [3.8] bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) #22748
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-10-18.16:00:16.227>
created_at = <Date 2020-10-07.10:08:31.925>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'datetime.time issue with pickling in PyPy'
updated_at = <Date 2020-10-18.16:00:16.227>
user = 'https://github.com/scaramallion'

    bugs.python.org fields:

    activity = <Date 2020-10-18.16:00:16.227>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-18.16:00:16.227>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2020-10-07.10:08:31.925>
creator = 'ddddaaaa'
dependencies = []
files = []
hgrepos = []
issue_num = 41966
keywords = ['patch']
message_count = 5.0
messages = ['378155', '378214', '378871', '378874', '378875']
nosy_count = 5.0
nosy_names = ['belopolsky', 'serhiy.storchaka', 'p-ganssle', 'miss-islington', 'ddddaaaa']
pr_nums = ['22731', '22747', '22748']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41966'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
    @scaramallion
    Copy link
    Mannequin Author

    scaramallion mannequin commented Oct 7, 2020

    I've run into an issue pickling a datetime.time subclass in PyPy3. I believe it arises because PyPy uses the pure Python implementation of time and time.__reduce_ex__() returns (time, ...) on this line (

    cpython/Lib/datetime.py

    Line 1551 in 044a104

    return (time, self._getstate(protocol))
    ) rather than (self.__class__, ...) as it does for datetime.datetime (and in datetime.date.__reduce__()). So when pickle creates the dump it uses the time class rather than my subclass.

    @scaramallion scaramallion mannequin added 3.7 (EOL) end of life 3.10 only security fixes 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 7, 2020
    @scaramallion
    Copy link
    Mannequin Author

    scaramallion mannequin commented Oct 8, 2020

    Code demonstrating the issue; the CTimeFoo class is pickled correctly, but TimeFoo isn't.

    import builtins
from _datetime import time as ctime

original_importer = builtins.__import__

def my_importer(name, globals, locals, fromlist, level):
    if name == '_datetime':
        raise ImportError

    return original_importer(name, globals, locals, fromlist, level)

builtins.__import__ = my_importer

import datetime

builtins.__import__ = original_importer

import pickle


class CTimeFoo(ctime): pass
class TimeFoo(datetime.time): pass
class DateFoo(datetime.date): pass


if __name__ == "__main__":
    t = DateFoo(2001, 2, 3) 
    d = pickle.dumps(t)  # OK
    print(d)
    # b'\x80\x04\x95#\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x07DateFoo\x94\x93\x94C\x04\x07\xd1\x02\x03\x94\x85\x94R\x94.'
    t = pickle.loads(d)

    t = CTimeFoo(1, 2, 3)  
    d = pickle.dumps(t)  # OK
    print(d)
    # b'\x80\x04\x95&\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x08CTimeFoo\x94\x93\x94C\x06\x01\x02\x03\x00\x00\x00\x94\x85\x94R\x94.'
    t = pickle.loads(d)

    t = TimeFoo(1, 2, 3)
    d = pickle.dumps(t)  # :(
    print(d)
    # b'\x80\x04\x95"\x00\x00\x00\x00\x00\x00\x00\x8c\x08datetime\x94\x8c\x04time\x94\x93\x94C\x06\x01\x02\x03\x00\x00\x00\x94\x85\x94R\x94.'
    t = pickle.loads(d)

    @serhiy-storchaka serhiy-storchaka removed 3.7 (EOL) end of life labels Oct 8, 2020
    @serhiy-storchaka
    Copy link
    Member

    New changeset c304c9a by scaramallion in branch 'master':
    bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731)
    c304c9a

    @serhiy-storchaka
    Copy link
    Member

    New changeset a055ced by Miss Skeleton (bot) in branch '3.9':
    bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) (GH-22747)
    a055ced

    @serhiy-storchaka
    Copy link
    Member

    New changeset 1040299 by Miss Skeleton (bot) in branch '3.8':
    bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) (GH-22748)
    1040299

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @serhiy-storchaka