Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancements/fixes to pure-python datetime module #65057

Closed
bdkearns mannequin opened this issue Mar 6, 2014 · 18 comments
Closed

Enhancements/fixes to pure-python datetime module #65057

bdkearns mannequin opened this issue Mar 6, 2014 · 18 comments
Assignees
Labels
type-bug An unexpected behavior, bug, or error

Comments

@bdkearns
Copy link
Mannequin

bdkearns mannequin commented Mar 6, 2014

BPO 20858
Nosy @malemburg, @tim-one, @abalkin, @benjaminp, @MojoVampire
Files
  • datetime-py34.patch
  • datetime-py33.patch
  • datetime-py33-v2.patch
  • datetime-py34-v2.patch
  • datetime-py33-v3.patch
  • datetime-py34-v3.patch
  • datetime-py35.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/abalkin'
    closed_at = <Date 2014-09-29.01:58:29.804>
    created_at = <Date 2014-03-06.18:37:38.860>
    labels = ['type-bug']
    title = 'Enhancements/fixes to pure-python datetime module'
    updated_at = <Date 2014-09-29.14:56:35.673>
    user = 'https://bugs.python.org/bdkearns'

    bugs.python.org fields:

    activity = <Date 2014-09-29.14:56:35.673>
    actor = 'berker.peksag'
    assignee = 'belopolsky'
    closed = True
    closed_date = <Date 2014-09-29.01:58:29.804>
    closer = 'benjamin.peterson'
    components = []
    creation = <Date 2014-03-06.18:37:38.860>
    creator = 'bdkearns'
    dependencies = []
    files = ['34293', '34294', '34295', '34296', '34297', '34298', '35894']
    hgrepos = []
    issue_num = 20858
    keywords = ['patch']
    message_count = 18.0
    messages = ['212832', '212833', '212840', '212846', '212850', '212852', '212854', '212855', '212856', '212857', '212858', '221907', '222515', '222942', '222944', '222946', '222947', '227780']
    nosy_count = 7.0
    nosy_names = ['lemburg', 'tim.peters', 'belopolsky', 'benjamin.peterson', 'python-dev', 'bdkearns', 'josh.r']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue20858'
    versions = ['Python 3.5']

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 6, 2014

    This patch brings the pure-python datetime more in-line with the C module. We have been running these modifications in PyPy2 stdlib for more than a year with no issue.

    Includes:

    • General PEP8/cleanups
    • Better testing of argument types passed to constructors
    • Removal of duplicate operations (in some paths values were checked twice for validity)
    • Optimization of timedelta creation (brings it from 8-9usec to ~6 usec on CPython 3.3 on local machine)
    • Enhancements/bug fixes in tests

    @bdkearns bdkearns mannequin added the type-bug An unexpected behavior, bug, or error label Mar 6, 2014
    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 6, 2014

    Also includes bug fixes/tests for certain rounding cases (doesn't apply to the 3.4 version).

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 6, 2014

    Updated patch to v2 with another test/fix for type checking of __format__ argument to match the C module.

    @abalkin abalkin self-assigned this Mar 6, 2014
    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Mar 6, 2014

    _check_int_field seems needlessly complex. When you want a value that is logically an integer (not merely capable of being coerced to an integer), you want object.__index__, per PEP-357, or to avoid explicit calls to special methods, use operator.index. Any reason to not just use operator.index directly?

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 6, 2014

    The C datetime module uses the 'i' code for parsing these args, not 'n' (which would correspond to operator.index). Using operator.index fails a test case I added (cases for classes like decimal.Decimal, implementing __int__ but not __index__).

    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Mar 6, 2014

    That's actually an argument to fix the C datetime implementation. Right now, you get:

        >>> from decimal import Decimal as d
        >>> from datetime import datetime
        >>> datetime(d("2000.5"), 1, 2)
        datetime.datetime(2000, 1, 2, 0, 0)

    This is wildly inconsistent; if you passed 2000.0, it would raise an exception because float (even floats directly equivalent to an int value) are forbidden. But the logically equivalent Decimal type will work just fine, silently truncating. Basically any user defined type with integer coercion (but not integer equivalence) would have the same problem; str doesn't, because str is special cased (it doesn't actually have __int__), but any user-defined str-like class that defined int coercion would work as a datetime arg in a way str does not.

    You've just given me an excuse to open my first bug. Thanks! :-)

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 6, 2014

    Right, that's the behavior as it stands, so I hope this patch can be considered independently of that issue (and if such a change is made to the C implementation, then a corresponding change could be made in the python implementation).

    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented Mar 7, 2014

    Oh, definitely. No reason to delay this just because I have my knickers in a twist on a tangential matter.

    @abalkin
    Copy link
    Member

    abalkin commented Mar 7, 2014

    I would like to hear from PyPy developers before we decide what to do with this effort. Pure Python implementation is not used by CPython,
    but I am afraid that people who actually use it will not appreciate the code churn.

    @abalkin
    Copy link
    Member

    abalkin commented Mar 7, 2014

    Oh - I did not realize that this originated in PyPy.

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Mar 7, 2014

    Yes, I am the PyPy developer who worked on these datetime improvements there -- just finally got around to pushing them upstream.

    @abalkin
    Copy link
    Member

    abalkin commented Jun 29, 2014

    Brian,

    I would like to apply your changes for 3.5. Do you have any updates?

    @bdkearns
    Copy link
    Mannequin Author

    bdkearns mannequin commented Jul 7, 2014

    Updated patch, now it also caches the result of __hash__ like the C accelerator.

    @abalkin
    Copy link
    Member

    abalkin commented Jul 13, 2014

    Brian,

    Could you, please update the summary of your changes from your first post? For example, you did not mention caching of the timedelta hashes.

    This particular chance seems to call for a discussion.

    Do we cache timedelta hashes in C implementation? What is the general wisdom on this technique? AFAICR, such hashing is done in integer objects, but I vaguely remember an old discussion on whether the same should be done for tuples. Can you remind me what was the outcome for tuples?

    @abalkin
    Copy link
    Member

    abalkin commented Jul 13, 2014

    Updated patch, now it also caches the result of __hash__ like the C accelerator.

    I should read your notes! Sorry, Brian. You already answered my questions. Too bad that the latest notes are so far from the entry area.

    Still, it would be helpful if you could provide a self-contained description that I can copy to the NEWS file. (Don't put it in the patch - NEWS file gets out of date very quickly - put it in a tracker comment.)

    Also, with your patch, are we in sync with PyPy? If so, with what version?

    @abalkin
    Copy link
    Member

    abalkin commented Jul 13, 2014

    [Josh Rosenberg]

    You've just given me an excuse to open my first bug.

    Did you open an issue for that? (Use "n" code in date/datetime constructors.)

    @abalkin
    Copy link
    Member

    abalkin commented Jul 13, 2014

    Here is the tuple hash caching thread that I mentioned above:

    https://mail.python.org/pipermail/python-dev/2003-August/037416.html

    Since the C code uses caching already, I don't think we need to discuss it any further. And the thread on tuples does not give any good reason not to cache anyways.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Sep 28, 2014

    New changeset 5313b4c0bb6c by Alexander Belopolsky in branch 'default':
    Closes issue bpo-20858: Enhancements/fixes to pure-python datetime module
    https://hg.python.org/cpython/rev/5313b4c0bb6c

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants