Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Direct sub-classing of pathlib.Path #68320

Closed
projetmbc mannequin opened this issue May 6, 2015 · 30 comments · Fixed by #31691
Closed

Direct sub-classing of pathlib.Path #68320

projetmbc mannequin opened this issue May 6, 2015 · 30 comments · Fixed by #31691
Labels
3.11 only security fixes stdlib Python modules in the Lib dir topic-pathlib type-feature A feature request or enhancement

Comments

@projetmbc
Copy link
Mannequin

projetmbc mannequin commented May 6, 2015

BPO 24132
Nosy @pfmoore, @pitrou, @keithy, @qb-cea, @miss-islington, @FFY00, @barneygale, @Xtrem532, @nyuszika7h, @kfollstad
PRs
  • bpo-24132: refactor pathlib to allow easy subclassing #6248
  • bpo-40107: stop using os.open() to implement pathlib.Path.open() #25240
  • bpo-42998: add 'user' parameter to pathlib.Path.home() #25271
  • bpo-43012: remove pathlib._Accessor #25701
  • bpo-44136: remove pathlib._Flavour #26141
  • bpo-24132: Direct sub-classing of pathlib.Path #26438
  • bpo-44412: add os.path.fileuri() function. #26708
  • bpo-24132: Add direct subclassing of PurePath/Path in pathlib #26906
  • bpo-24132: Add pathlib._AbstractPath #31085
  • gh-68320, gh-88302 - Allow for pathlib.Path subclassing #31691
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2015-05-06.05:26:53.017>
    labels = ['type-feature', 'library', '3.11']
    title = 'Direct sub-classing of pathlib.Path'
    updated_at = <Date 2022-03-09.23:54:09.299>
    user = 'https://bugs.python.org/projetmbc'

    bugs.python.org fields:

    activity = <Date 2022-03-09.23:54:09.299>
    actor = 'barneygale'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2015-05-06.05:26:53.017>
    creator = 'projetmbc'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 24132
    keywords = ['patch']
    message_count = 28.0
    messages = ['242643', '242651', '242656', '242657', '242689', '242696', '242699', '242700', '242701', '242702', '242703', '242705', '246007', '305799', '305811', '305827', '305830', '305918', '310634', '314561', '314582', '314625', '365277', '381321', '392695', '401946', '412354', '414821']
    nosy_count = 15.0
    nosy_names = ['paul.moore', 'pitrou', 'bronger', 'elguavas', 'piotr.dobrogost', 'Kevin.Norris', 'projetmbc', 'keithy', 'qb-cea', 'miss-islington', 'FFY00', 'barneygale', 'Xtrem532', 'nyuszika7h', 'kfollstad']
    pr_nums = ['6248', '25240', '25271', '25701', '26141', '26438', '26708', '26906', '31085', '31691']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue24132'
    versions = ['Python 3.11']

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    Hello.

    I have noticed a problem with the following code.

    from pathlib import Path
    
    class PPath(Path):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
    test = PPath("dir", "test.txt")

    This gives the following error message.

     
    Traceback (most recent call last):
      File "/Users/projetmbc/test.py", line 14, in <module>
        test = PPath("dir", "test.txt")
      File "/anaconda/lib/python3.4/pathlib.py", line 907, in __new__
        self = cls._from_parts(args, init=False)
      File "/anaconda/lib/python3.4/pathlib.py", line 589, in _from_parts
        drv, root, parts = self._parse_args(args)
      File "/anaconda/lib/python3.4/pathlib.py", line 582, in _parse_args
        return cls._flavour.parse_parts(parts)
    AttributeError: type object 'PPath' has no attribute '_flavour'

    This breaks the sub-classing from Python point of view.

    There is an ugly hack to sub-class Path but it's a bit unpythonic.

    @projetmbc projetmbc mannequin added the type-bug An unexpected behavior, bug, or error label May 6, 2015
    @projetmbc projetmbc mannequin changed the title Direct sub-classing of pathless.Path Direct sub-classing of pathlib.Path May 6, 2015
    @pfmoore
    Copy link
    Member

    pfmoore commented May 6, 2015

    One issue with your code - what would you expect str(test) to produce? "dir/test.txt" or "dir\test.txt"? That's the point of the "flavour" - is it a Windows path or a Unix path?

    Agreed that an easier method of creating Path subclasses that handle this type of thing would be useful, but any solution needs to make sure that developers don't overlook the Windows vs Unix implications.

    Can you give an actual use case (as opposed to the toy example)?

    @pitrou
    Copy link
    Member

    pitrou commented May 6, 2015

    The Path classes were not designed to be subclassable by the user.
    I'm not against making subclassing easier, but someone will have to propose a viable approach for that.

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    Hello.

    I will give a real example in 5 hours after my job. I will try tomorrow a
    solution to ease the subclassing using another dedicazted class PathPlus,
    sorry for the name. The idea would be to use this new class for
    customization, and also to define WindowsPath and PosixPath sub-classing
    this new class. By default PathPlus would be an empty class. I do not know
    if this works well. Maybe my idea is a bad one.

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 13:05 GMT+02:00 Antoine Pitrou <report@bugs.python.org>:

    Antoine Pitrou added the comment:

    The Path classes were not designed to be subclassable by the user.
    I'm not against making subclassing easier, but someone will have to
    propose a viable approach for that.

    ----------
    versions: +Python 3.5 -Python 3.4


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue24132\>


    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    Here are for example two extra methods that I have implemented.

    def __sub__(cls, path):
        """
    This magic method allows to use ``onepath - anotherpath`` instead of the
    long
    version ``onepath.relative_to(anotherpath)`` given by ``pathlib.Path``.
        """
        return cls.relative_to(path)
    
    def _ppath_common_with(cls, paths):
        """
    This method returns the path of the smaller common "folder" of the current
    path
    and at least one paths.

    python::
    from mistool import os_use

        path   = os_use.PPath("/Users/projects/source/doc")
        path_1 = os_use.PPath("/Users/projects/README")
        path_2 = os_use.PPath("/Users/projects/source/misTool/os_use.py")
    
        print(path.common_with((path_1, path_2)))
        """
        if not isinstance(paths, (list, tuple)):
            paths = [paths]
    
        commonparts = list(cls.parts)
    
        for onepath in paths:
            i = 0
    
            for common, actual in zip(commonparts, onepath.parts):
                if common == actual:
                    i += 1
                else:
                    break
    
            commonparts = commonparts[:i]
    
            if not commonparts:
                break
    
        commonpath = pathlib.Path("")
    
        for part in commonparts:
            commonpath /= part
    
        return commonpath

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 14:13 GMT+02:00 Christophe BAL <report@bugs.python.org>:

    Christophe BAL added the comment:

    Hello.

    I will give a real example in 5 hours after my job. I will try tomorrow a
    solution to ease the subclassing using another dedicazted class PathPlus,
    sorry for the name. The idea would be to use this new class for
    customization, and also to define WindowsPath and PosixPath sub-classing
    this new class. By default PathPlus would be an empty class. I do not know
    if this works well. Maybe my idea is a bad one.

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 13:05 GMT+02:00 Antoine Pitrou <report@bugs.python.org>:

    >
    > Antoine Pitrou added the comment:
    >
    > The Path classes were not designed to be subclassable by the user.
    > I'm not against making subclassing easier, but someone will have to
    > propose a viable approach for that.
    >
    > ----------
    > versions: +Python 3.5 -Python 3.4
    >
    > _______________________________________
    > Python tracker <report@bugs.python.org>
    > <http://bugs.python.org/issue24132\>
    > _______________________________________
    >

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue24132\>


    @pfmoore
    Copy link
    Member

    pfmoore commented May 6, 2015

    For that type of function, I'd suggest you use a standalone function rather than subclassing and methods or operator overloading. You don't gain enough to be worth the complexity of having to subclass path objects. And duck typing means that your function works for any subclass of (Pure)Path without change.

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    I don't agree with you. I prefer to add new functionalities to the paths I
    use. This is the power of OOP. It is easier and cleaner to use
    *mypath.common_with(otherpath)* than *common_with(*mypath, **other path)
    .

    Python is highly OOP, so you can't say *"don't use subclassing in your
    case"*. As a user, I should have the possibility to use the method I want.

    Another example is the use of *onepath - anotherpath* instead of
    *onepath.relative_to(*another path) . That's the power of the magic
    method to add this kind of feature.

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 20:21 GMT+02:00 Paul Moore <report@bugs.python.org>:

    Paul Moore added the comment:

    For that type of function, I'd suggest you use a standalone function
    rather than subclassing and methods or operator overloading. You don't gain
    enough to be worth the complexity of having to subclass path objects. And
    duck typing means that your function works for any subclass of (Pure)Path
    without change.

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue24132\>


    @pfmoore
    Copy link
    Member

    pfmoore commented May 6, 2015

    I have no problem with that - it's a style choice certainly.

    As I said, I'd like to see simpler subclassing of pathlib objects. I just think it'll be quite hard to do (given the complexities of classes for Windows/Unix as well as pure and concrete paths). So if it's just about examples like this, I personally would take the easier route and just go with standalone functions. If someone else felt strongly enough to design and implement a subclassing solution, that's fine though.

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    Are you the author of path lib ?

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 21:01 GMT+02:00 Paul Moore <report@bugs.python.org>:

    Paul Moore added the comment:

    I have no problem with that - it's a style choice certainly.

    As I said, I'd like to see simpler subclassing of pathlib objects. I just
    think it'll be quite hard to do (given the complexities of classes for
    Windows/Unix as well as pure and concrete paths). So if it's just about
    examples like this, I personally would take the easier route and just go
    with standalone functions. If someone else felt strongly enough to design
    and implement a subclassing solution, that's fine though.

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue24132\>


    @pfmoore
    Copy link
    Member

    pfmoore commented May 6, 2015

    Are you the author of path lib ?

    Nope, that's Antoine.

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented May 6, 2015

    OK.
    I will try to find a way to achieve an easier and cleaner way to sub class
    pathlib.Path and co.

    What is the good way to propose a patch ?

    *Christophe BAL*
    *Enseignant de mathématiques en Lycée **et développeur Python amateur*
    *---*
    *French math teacher in a "Lycée" **and **Python **amateur developer*

    2015-05-06 21:09 GMT+02:00 Paul Moore <report@bugs.python.org>:

    Paul Moore added the comment:

    > Are you the author of path lib ?

    Nope, that's Antoine.

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue24132\>


    @pfmoore
    Copy link
    Member

    pfmoore commented May 6, 2015

    What is the good way to propose a patch ?

    If you have a patch, attach it here, and it will get reviewed.

    @KevinNorris
    Copy link
    Mannequin

    KevinNorris mannequin commented Jul 1, 2015

    If I were designing pathlib from scratch, I would not have a separate Path class. I would instead do something like this:

    In pathlib.py:

        if os.name == 'nt':
            Path = WindowsPath
        else:
            Path = PosixPath

    Alternatively, Path() could be a factory function that picks one of those classes at runtime.

    Of course, that still leaves the issue of where to put the method implementations which currently live in Path. We could change the name of Path to _Path and use the code above to continue providing a Path alias, but that might be too confusing. Another possibility is to pull those methods out into top-level functions and then alias them into methods in WindowsPath and PosixPath (perhaps using a decorator-like-thing to pass the flavor, instead of attaching it to the class).

    The main thing, though, is that Path should not depend on its subclasses. That really strikes me as poor design, since it produces issues like this one.

    @elguavas
    Copy link
    Mannequin

    elguavas mannequin commented Nov 8, 2017

    Using a set of paths with special properties and formats in a project, thought "the cleanest oop way to do this is try out python's oop paths in pathlib". Subclassed Path to implement my extra (non platfor specific) properties and fell at the first hurdle because of this issue...

    for me pathlib does not provide oop paths if i can't subclass Path, for whatever reason.

    reverted to treating paths as strings and writing functions to handle my special path properties and formats.

    i was also surprised when i found another bug report on this issue that said it was closed for 3.7, great i thought this has been solved, but no, the other report was closed because it was about the same issue as this ancient report.

    @pfmoore
    Copy link
    Member

    pfmoore commented Nov 8, 2017

    @elguavas the problem is, no-one has proposed a patch. There's not likely to be much movement on this until someone provides one.

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented Nov 8, 2017

    For the moment, you can take a look at this little script that acheives
    subclassing of Path :
    https://github.com/bc-python/mistool/blob/master/mistool/os_use.py
    (search for class Path).

    Le 08/11/2017 à 09:55, Paul Moore a écrit :

    Paul Moore <p.f.moore@gmail.com> added the comment:

    @elguavas the problem is, no-one has proposed a patch. There's not likely to be much movement on this until someone provides one.

    ----------


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue24132\>


    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented Nov 8, 2017

    Mistyping : /search for class PPath/ with two P.

    Le 08/11/2017 à 13:59, Christophe BAL a écrit :

    Christophe BAL <projetmbc@gmail.com> added the comment:

    For the moment, you can take a look at this little script that acheives
    subclassing of Path :
    https://github.com/bc-python/mistool/blob/master/mistool/os_use.py
    (search for class Path).

    Le 08/11/2017 à 09:55, Paul Moore a écrit :
    > Paul Moore <p.f.moore@gmail.com> added the comment:
    >
    > @elguavas the problem is, no-one has proposed a patch. There's not likely to be much movement on this until someone provides one.
    >
    > ----------
    >
    > _______________________________________
    > Python tracker <report@bugs.python.org>
    > <https://bugs.python.org/issue24132\>
    > _______________________________________
    ----------


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue24132\>


    @elguavas
    Copy link
    Mannequin

    elguavas mannequin commented Nov 8, 2017

    @paul.moore is the original contributor mia? i seem to remember pathlib as once being marked 'provisional', i think it should have stayed that way until this problem was resolved. easy to say i know ;) when i don't have a patch.

    @projetmbc yes i found various work-arounds on the web and decided to not use any of them. really i feel this should be fixed as it's a jarring inconsistency with naturally expected behaviour for a class in python.

    so i added my report to this as a topic bump because i don't think this should be forgotten about and in case anyone might come up with an idea how to fix it.

    @keithy
    Copy link
    Mannequin

    keithy mannequin commented Jan 24, 2018

    Look at the architecture of Rio in Ruby (also ported to Squeak/Smalltalk)

    Leave Path to handle path stuff, and have another class to handle Platform stuff.

    https://rubygems.org/gems/rio/versions/0.6.0

    @qb-cea
    Copy link
    Mannequin

    qb-cea mannequin commented Mar 28, 2018

    Hi all,

    I made a pull request proposing a fix for this issue. There is still quite a lot to be done:

    • I exposed some variables (and probably methods too) that used to be hidden;
    • I did not update the documentation;
    • I did not add a proper test.

    I will try to fix those by the end of the week.

    The patch mainly consists of two things:

    • having Path (resp. PurePath) be a variable pointing at either (Pure)PosixPath or (Pure)WindowsPath, depending on the platform (like Kevin Norris suggested);
    • introducing two new abstract classes _PurePath and ConcretePath from which PurePosixPath, PureWindowsPath and PosixPath, WindowsPath classes inherit;
    • removing the _Flavor classes, and redistributing their method to platform-specific classes.

    Ideally I would like _PurePath to become a public class, but I could not come up with a proper name. Any feedback is more than welcome =]

    @projetmbc
    Copy link
    Mannequin Author

    projetmbc mannequin commented Mar 28, 2018

    Hello.

    What about AbstractPath instead of _PurePath ?

    Le 28/03/2018 à 02:30, qb-cea a écrit :

    qb-cea <quentin.bouget@cea.fr> added the comment:

    Hi all,

    I made a pull request proposing a fix for this issue. There is still quite a lot to be done:

    • I exposed some variables (and probably methods too) that used to be hidden;
    • I did not update the documentation;
    • I did not add a proper test.

    I will try to fix those by the end of the week.

    The patch mainly consists of two things:

    • having Path (resp. PurePath) be a variable pointing at either (Pure)PosixPath or (Pure)WindowsPath, depending on the platform (like Kevin Norris suggested);
    • introducing two new abstract classes _PurePath and ConcretePath from which PurePosixPath, PureWindowsPath and PosixPath, WindowsPath classes inherit;
    • removing the _Flavor classes, and redistributing their method to platform-specific classes.

    Ideally I would like _PurePath to become a public class, but I could not come up with a proper name. Any feedback is more than welcome =]

    ----------


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue24132\>


    @qb-cea
    Copy link
    Mannequin

    qb-cea mannequin commented Mar 29, 2018

    What about AbstractPath instead of _PurePath ?

    I will use this, thanks.

    @barneygale
    Copy link
    Mannequin

    barneygale mannequin commented Mar 29, 2020

    I'm taking another look at making pathlib extensible. There's some discussion here: https://discuss.python.org/t/make-pathlib-extensible/3428

    List or preparatory bugfixes and tidy-ups: https://docs.google.com/spreadsheets/d/1TicFDMudKKA6CZcrscg1Xq9kt5Q8To8y0hADGw9u11I/edit#gid=0

    @qb-cea
    Copy link
    Mannequin

    qb-cea mannequin commented Nov 18, 2020

    Hi,

    Thanks for reviving this! Feel free to reuse any code I wrote in my PR (or the whole PR itself), I do not think I will ever get around to finishing this work myself.

    @iritkatriel iritkatriel added 3.10 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels Mar 26, 2021
    @barneygale
    Copy link
    Mannequin

    barneygale mannequin commented May 2, 2021

    Progress report:

    I've been working on tidying up the pathlib internals over the 3.9 and 3.10 releases. We're now in a position where:

    • pathlib._Flavour is entirely pure, and doesn't make any os calls
    • pathlib._Accessor handles all os access.

    The internal abstractions are now much tighter, which allows us to begin refactoring them with confidence!

    The next step is to remove accessors, in bpo-43012.

    After that I'll finally be in a position to start working on this bug!

    @nyuszika7h
    Copy link
    Mannequin

    nyuszika7h mannequin commented Sep 16, 2021

    I agree this would be nice. For now, I'm doing this as a hack:

    class Path(type(pathlib.Path())):
        ...

    @merwok merwok added 3.11 only security fixes and removed 3.10 only security fixes labels Jan 1, 2022
    @miss-islington
    Copy link
    Contributor

    New changeset 08f8301 by Barney Gale in branch 'main':
    bpo-43012: remove pathlib._Accessor (GH-25701)
    08f8301

    @barneygale
    Copy link
    Mannequin

    barneygale mannequin commented Mar 9, 2022

    If/when python/issues-test-cpython#31691 lands, I think this bug can be resolved: the original repro case will no longer raise AttributeError, and subclasses will be able to customize behaviour without needing to define further "flavour" or "accessor" subclasses.

    @0xallie
    Copy link

    0xallie commented Sep 6, 2022

    I agree this would be nice. For now, I'm doing this as a hack:

    class Path(type(pathlib.Path())):
        ...

    Worth noting that this confuses mypy even when adding # type: ignore, so now I'm doing this instead:

    import pathlib
    from typing import TYPE_CHECKING
    
    # We can't subclass pathlib.Path directly (https://github.com/python/cpython/issues/68320)
    if TYPE_CHECKING:
        class BasePath(pathlib.Path):
            pass
    else:
        class BasePath(type(pathlib.Path())):
            pass
    
    
    class Path(BasePath):
       # Actual custom path class goes here

    brettcannon added a commit that referenced this issue Dec 23, 2022
    …-31691)
    
    Users may wish to define subclasses of `pathlib.Path` to add or modify
    existing methods. Before this change, attempting to instantiate a subclass
    raised an exception like:
    
        AttributeError: type object 'PPath' has no attribute '_flavour'
    
    Previously the `_flavour` attribute was assigned as follows:
    
        PurePath._flavour        = xxx not set!! xxx
        PurePosixPath._flavour   = _PosixFlavour()
        PureWindowsPath._flavour = _WindowsFlavour()
    
    This change replaces it with a `_pathmod` attribute, set as follows:
    
        PurePath._pathmod        = os.path
        PurePosixPath._pathmod   = posixpath
        PureWindowsPath._pathmod = ntpath
    
    Functionality from `_PosixFlavour` and `_WindowsFlavour` is moved into
    `PurePath` as underscored-prefixed classmethods. Flavours are removed.
    
    Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
    Co-authored-by: Brett Cannon <brett@python.org>
    Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
    Co-authored-by: Eryk Sun <eryksun@gmail.com>
    iritkatriel added a commit to iritkatriel/cpython that referenced this issue Dec 28, 2022
    * Correct CVE-2020-10735 documentation (python#100306)
    
    * pythongh-94912: Added marker for non-standard coroutine function detection (python#99247)
    
    This introduces a new decorator `@inspect.markcoroutinefunction`,
    which, applied to a sync function, makes it appear async to
    `inspect.iscoroutinefunction()`.
    
    * Docs: Don't upload CI artifacts (python#100330)
    
    * pythongh-89727: Fix os.walk RecursionError on deep trees (python#99803)
    
    Use a stack to implement os.walk iteratively instead of recursively to
    avoid hitting recursion limits on deeply nested trees.
    
    * pythongh-69929: re docs: Add more specific definition of \w (python#92015)
    
    Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
    
    * pythongh-89051: Add ssl.OP_LEGACY_SERVER_CONNECT (python#93927)
    
    Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
    Co-authored-by: Christian Heimes <christian@python.org>
    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    Fixes python#89051
    
    * pythongh-88211: Change lower-case and upper-case to match recommendations in imaplib docs (python#99625)
    
    * pythongh-100348: Fix ref cycle in `asyncio._SelectorSocketTransport` with `_read_ready_cb` (python#100349)
    
    * pythongh-99925: Fix inconsistency in `json.dumps()` error messages (pythonGH-99926)
    
    * Clarify that every thread has its own default context in contextvars (python#99246)
    
    * pythongh-99576: Fix cookiejar file that was not truncated for some classes (pythonGH-99616)
    
    Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    
    * pythongh-100188: Reduce misses in BINARY_SUBSCR_(LIST/TUPLE)_INT (python#100189)
    
    Don't specialize if the index is negative.
    
    * pythongh-99991: improve docs on str.encode and bytes.decode (python#100198)
    
    Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
    
    * pythongh-91081: Add note on WeakKeyDictionary behavior when deleting a replaced entry (python#91499)
    
    Co-authored-by: Pieter Eendebak <P.T.eendebak@tudelft.nl>
    Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
    
    * pythongh-85267: Improvements to inspect.signature __text_signature__ handling (python#98796)
    
    This makes a couple related changes to inspect.signature's behaviour
    when parsing a signature from `__text_signature__`.
    
    First, `inspect.signature` is documented as only raising ValueError or
    TypeError. However, in some cases, we could raise RuntimeError.  This PR
    changes that, thereby fixing python#83685.
    
    (Note that the new ValueErrors in RewriteSymbolics are caught and then
    reraised with a message)
    
    Second, `inspect.signature` could randomly drop parameters that it
    didn't understand (corresponding to `return None` in the `p` function).
    This is the core issue in python#85267. I think this is very surprising
    behaviour and it seems better to fail outright.
    
    Third, adding this new failure broke a couple tests. To fix them (and to
    e.g. allow `inspect.signature(select.epoll.register)` as in python#85267), I
    add constant folding of a couple binary operations to RewriteSymbolics.
    
    (There's some discussion of making signature expression evaluation
    arbitrary powerful in python#68155. I think that's out of scope. The
    additional constant folding here is pretty straightforward, useful, and
    not much of a slippery slope)
    
    Fourth, while python#85267 is incorrect about the cause of the issue, it turns
    out if you had consecutive newlines in __text_signature__, you'd get
    `tokenize.TokenError`.
    
    Finally, the `if name is invalid:` code path was dead, since
    `parse_name` never returned `invalid`.
    
    * pythonGH-100363: Speed up `asyncio.get_running_loop` (python#100364)
    
    * pythonGH-100133: fix `asyncio` subprocess losing `stderr` and `stdout` output (python#100154)
    
    * pythongh-100374: Fixed a bug in socket.getfqdn() (pythongh-100375)
    
    * pythongh-100129: Add tests for pickling all builtin types and functions (pythonGH-100142)
    
    * Remove unused variable from `dis._find_imports` (python#100396)
    
    * pythongh-78878: Fix crash when creating an instance of `_ctypes.CField` (python#14837)
    
    * pythonGH-69564: Clarify use of octal format of mode argument in help(os.chmod) (python#20621)
    
    Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
    
    * pythonGH-99554: Pack location tables more effectively (pythonGH-99556)
    
    * Correct typo in typing.py (python#100423)
    
    In the docstring of `ParamSpec`, the name of `P = ParamSpec('P')` was
    mistakenly written as `'T'`.
    
    * pythongh-99761: Add `_PyLong_IsPositiveSingleDigit` function to check for single digit integers  (python#100064)
    
    * pythonGH-99770: Make the correct call specialization fail kind show up in the stats (pythonGH-99771)
    
    * pythongh-78997: fix bad rebase of moved test file (python#100424)
    
    * pythongh-100344: Add C implementation for `asyncio.current_task` (python#100345)
    
    Co-authored-by: pranavtbhat
    
    * pythonGH-99554: Trim trailing whitespace (pythonGH-100435)
    
    
    
    Automerge-Triggered-By: GH:brandtbucher
    
    * pythongh-85432: Harmonise parameter names between C and pure-Python implementations of `datetime.time.strftime`, `datetime.datetime.fromtimestamp` (python#99993)
    
    * pythongh-57762: fix misleading tkinter.Tk docstring (python#98837)
    
    Mentioned as a desired change by terryjreedy on the corresponding issue,
    since Tk is not a subclass of Toplevel.
    
    * pythongh-48496: Added example and link to faq for UnboundLocalError in reference (python#93068)
    
    * Fix typo in 3.12 What's New (python#100449)
    
    * pythongh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (pythonGH-5576)
    
    The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
    
    * pythonGH-100459: fix copy-paste errors in specialization stats (pythonGH-100460)
    
    * pythongh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (python#100182)
    
    * pythongh-98712: Clarify "readonly bytes-like object" semantics in C arg-parsing docs (python#98710)
    
    * pythongh-92216: improve performance of `hasattr` for type objects (pythonGH-99979)
    
    * pythongh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (pythonGH-100289)
    
    * Revert "pythongh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (pythonGH-100289)" (python#100468)
    
    This reverts commit c3c7848.
    
    * pythongh-94155: Reduce hash collisions for code objects (python#100183)
    
    * Uses a better hashing algorithm to get better dispersion and remove commutativity.
    
    * Incorporates `co_firstlineno`, `Py_SIZE(co)`, and bytecode instructions.
    
    * This is now the entire set of criteria used in `code_richcompare`, except for `_PyCode_ConstantKey` (which would incorporate the types of `co_consts` rather than just their values).
    
    * pythongh-83076: 3.8x speed improvement in (Async)Mock instantiation (python#100252)
    
    * pythongh-99482: remove `jython` compatibility parts from stdlib and tests (python#99484)
    
    * bpo-40447: accept all path-like objects in compileall.compile_file (python#19883)
    
    Signed-off-by: Filipe Laíns <lains@archlinux.org>
    Signed-off-by: Filipe Laíns <lains@riseup.net>
    Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
    Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
    
    * pythonGH-100425: Improve accuracy of builtin sum() for float inputs (pythonGH-100426)
    
    * pythongh-68320, pythongh-88302 - Allow for private `pathlib.Path` subclassing (pythonGH-31691)
    
    Users may wish to define subclasses of `pathlib.Path` to add or modify
    existing methods. Before this change, attempting to instantiate a subclass
    raised an exception like:
    
        AttributeError: type object 'PPath' has no attribute '_flavour'
    
    Previously the `_flavour` attribute was assigned as follows:
    
        PurePath._flavour        = xxx not set!! xxx
        PurePosixPath._flavour   = _PosixFlavour()
        PureWindowsPath._flavour = _WindowsFlavour()
    
    This change replaces it with a `_pathmod` attribute, set as follows:
    
        PurePath._pathmod        = os.path
        PurePosixPath._pathmod   = posixpath
        PureWindowsPath._pathmod = ntpath
    
    Functionality from `_PosixFlavour` and `_WindowsFlavour` is moved into
    `PurePath` as underscored-prefixed classmethods. Flavours are removed.
    
    Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
    Co-authored-by: Brett Cannon <brett@python.org>
    Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
    Co-authored-by: Eryk Sun <eryksun@gmail.com>
    
    * pythongh-99947: Ensure unreported errors are chained for SystemError during import (pythonGH-99946)
    
    * Add "strict" to dotproduct(). Add docstring. Factor-out common code. (pythonGH-100480)
    
    * pythongh-94808: improve test coverage of number formatting (python#99472)
    
    * pythongh-100454: Start running SSL tests with OpenSSL 3.1.0-beta1 (python#100456)
    
    * pythongh-100268: Add is_integer method to int (python#100439)
    
    This improves the lives of type annotation users of `float` - which type checkers implicitly treat as `int|float` because that is what most code actually wants. Before this change a `.is_integer()` method could not be assumed to exist on things annotated as `: float` due to the method not existing on both types.
    
    * pythongh-77771: Add enterabs example in sched (python#92716)
    
    Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
    
    * pythonGH-91166: Implement zero copy writes for `SelectorSocketTransport` in asyncio (python#31871)
    
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    
    * pythonGH-91166: Implement zero copy writes for `SelectorSocketTransport` in asyncio (python#31871)
    
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    
    * Misc Itertools recipe tweaks (pythonGH-100493)
    
    * pythongh-100357: Convert several functions in `bltinsmodule` to AC (python#100358)
    
    * Remove wrong comment about `repr` in `test_unicode` (python#100495)
    
    * pythongh-99908: Tutorial: Modernize the 'data-record class' example (python#100499)
    
    Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
    
    * pythongh-100474: Fix handling of dirs named index.html in http.server (pythonGH-100475)
    
    
    
    If you had a directory called index.html or index.htm within a directory, it would cause http.server to return a 404 Not Found error instead of the directory listing. This came about due to not checking that the index was a regular file.
    
    I have also added a test case for this situation.
    
    Automerge-Triggered-By: GH:merwok
    
    * pythongh-100287: Fix unittest.mock.seal with AsyncMock (python#100496)
    
    * pythongh-99535: Add test for inheritance of annotations and update documentation (python#99990)
    
    * pythongh-100428: Make float documentation more accurate (python#100437)
    
    Previously, the grammar did not accept `float("10")`.
    Also implement mdickinson's suggestion of removing the indirection.
    
    * [Minor PR] Quotes in documentation changed into code blocks (python#99536)
    
    Minor formatting fix in documentation
    
    Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
    
    * pythongh-100472: Fix docs claim that compileall parameters could be bytes (python#100473)
    
    * pythongh-100519: simplification to `eff_request_host` in cookiejar.py (python#99588)
    
    `IPV4_RE` includes a `.`, and the `.find(".") == -1` included here is already testing to make sure there's no dot, so this part of the expression is tautological. Instead use more modern `in` syntax to make it clear what the check is doing here. The simplified implementation more clearly matches the wording in RFC 2965.
    
    Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
    
    * pythongh-99308: Clarify re docs for byte pattern group names (python#99311)
    
    * pythongh-92446: Improve argparse choices docs; revert bad change to lzma docs (python#94627)
    
    Based on the definition of the collections.abc classes, it is more accurate to use "sequence" instead of "container" when describing argparse choices.
    
    A previous attempt at fixing this in python#92450 was mistaken; this PR reverts that change.
    
    Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
    
    * Fix name of removed `inspect.Signature.from_builtin` method in 3.11.0a2 changelog (python#100525)
    
    * pythongh-100520: Fix `rst` markup in `configparser`  docstrings (python#100524)
    
    * pythongh-99509: Add `__class_getitem__` to `multiprocessing.queues.Queue` (python#99511)
    
    * pythongh-94603: micro optimize list.pop (pythongh-94604)
    
    * Remove `NoneType` redefinition from `clinic.py` (python#100551)
    
    * pythongh-100553: Improve accuracy of sqlite3.Row iter test (python#100555)
    
    * pythonGH-98831: Modernize a ton of simpler instructions (python#100545)
    
    * load_const and load_fast aren't families for now
    * Don't decref unmoved names
    * Modernize GET_ANEXT
    * Modernize GET_AWAITABLE
    * Modernize ASYNC_GEN_WRAP
    * Modernize YIELD_VALUE
    * Modernize POP_EXCEPT (in more than one way)
    * Modernize PREP_RERAISE_STAR
    * Modernize LOAD_ASSERTION_ERROR
    * Modernize LOAD_BUILD_CLASS
    * Modernize STORE_NAME
    * Modernize LOAD_NAME
    * Modernize LOAD_CLASSDEREF
    * Modernize LOAD_DEREF
    * Modernize STORE_DEREF
    * Modernize COPY_FREE_VARS (mark it as done)
    * Modernize LIST_TO_TUPLE
    * Modernize LIST_EXTEND
    * Modernize SET_UPDATE
    * Modernize SETUP_ANNOTATIONS
    * Modernize DICT_UPDATE
    * Modernize DICT_MERGE
    * Modernize MAP_ADD
    * Modernize IS_OP
    * Modernize CONTAINS_OP
    * Modernize CHECK_EXC_MATCH
    * Modernize IMPORT_NAME
    * Modernize IMPORT_STAR
    * Modernize IMPORT_FROM
    * Modernize JUMP_FORWARD (mark it as done)
    * Modernize JUMP_BACKWARD (mark it as done)
    
    Signed-off-by: Filipe Laíns <lains@archlinux.org>
    Signed-off-by: Filipe Laíns <lains@riseup.net>
    Co-authored-by: Jeremy Paige <ucodery@gmail.com>
    Co-authored-by: Carlton Gibson <carlton@noumenal.es>
    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    Co-authored-by: Jon Burdo <jon@jonburdo.com>
    Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
    Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
    Co-authored-by: Thomas Grainger <tagrain@gmail.com>
    Co-authored-by: Brad Wolfe <brad.wolfe@gmail.com>
    Co-authored-by: Richard Kojedzinszky <rkojedzinszky@users.noreply.github.com>
    Co-authored-by: František Nesveda <fnesveda@users.noreply.github.com>
    Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
    Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
    Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
    Co-authored-by: Bisola Olasehinde <horlasehinde@gmail.com>
    Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
    Co-authored-by: Pieter Eendebak <P.T.eendebak@tudelft.nl>
    Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
    Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
    Co-authored-by: Dominic Socular <BBH@awsl.rip>
    Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    Co-authored-by: Hai Shi <shihai1992@gmail.com>
    Co-authored-by: amaajemyfren <32741226+amaajemyfren@users.noreply.github.com>
    Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
    Co-authored-by: david-why <david_why@outlook.com>
    Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
    Co-authored-by: penguin_wwy <940375606@qq.com>
    Co-authored-by: Eli Schwartz <eschwartz93@gmail.com>
    Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
    Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
    Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
    Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
    Co-authored-by: Bill Fisher <william.w.fisher@gmail.com>
    Co-authored-by: Petr Viktorin <encukou@gmail.com>
    Co-authored-by: Ken Jin <kenjin@python.org>
    Co-authored-by: Carl Meyer <carl@oddbird.net>
    Co-authored-by: Filipe Laíns <lains@riseup.net>
    Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
    Co-authored-by: Barney Gale <barney.gale@gmail.com>
    Co-authored-by: Brett Cannon <brett@python.org>
    Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
    Co-authored-by: Eryk Sun <eryksun@gmail.com>
    Co-authored-by: Sebastian Berg <sebastianb@nvidia.com>
    Co-authored-by: Illia Volochii <illia.volochii@gmail.com>
    Co-authored-by: JosephSBoyle <48555120+JosephSBoyle@users.noreply.github.com>
    Co-authored-by: James Frost <git@frost.cx>
    Co-authored-by: MonadChains <monadchains@gmail.com>
    Co-authored-by: Bart Broere <mail@bartbroere.eu>
    Co-authored-by: Glyph <code@glyph.im>
    Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
    Co-authored-by: Ilya Kulakov <kulakov.ilya@gmail.com>
    Co-authored-by: Guy Yagev <yourlefthandman8@gmail.com>
    Co-authored-by: Jakub Kuczys <me@jacken.men>
    @mzipay
    Copy link

    mzipay commented Apr 6, 2023

    Sorry if I'm "late to the party," but ALL of the discussion from non-mannequins seems (to me) to miss the point entirely.

    The pathlib module fails (grossly, IMO) to respect two key "Pythonic" principles:
    #. If the implementation is hard to explain, it's a bad idea.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes stdlib Python modules in the Lib dir topic-pathlib type-feature A feature request or enhancement
    Projects
    None yet
    Development

    Successfully merging a pull request may close this issue.

    7 participants