Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

len(platform.uname()) has changed in Python 3.9 #84750

Closed
tucked mannequin opened this issue May 8, 2020 · 13 comments
Closed

len(platform.uname()) has changed in Python 3.9 #84750

tucked mannequin opened this issue May 8, 2020 · 13 comments
Labels
3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tucked
Copy link
Mannequin

tucked mannequin commented May 8, 2020

BPO 40570
Nosy @malemburg, @jaraco, @tucked
PRs
  • bpo-40570: Count the processor property in len(platform.uname()) #20009
  • bpo-40570: Improve compatibility of uname_result with late-bound .platform #20015
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-05-09.14:14:14.194>
    created_at = <Date 2020-05-08.20:02:20.834>
    labels = ['type-bug', 'library', '3.9']
    title = 'len(platform.uname()) has changed in Python 3.9'
    updated_at = <Date 2020-05-13.07:47:25.587>
    user = 'https://github.com/tucked'

    bugs.python.org fields:

    activity = <Date 2020-05-13.07:47:25.587>
    actor = 'lemburg'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-05-09.14:14:14.194>
    closer = 'jaraco'
    components = ['Library (Lib)']
    creation = <Date 2020-05-08.20:02:20.834>
    creator = 'tucked'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40570
    keywords = ['3.9regression']
    message_count = 11.0
    messages = ['368459', '368486', '368487', '368491', '368502', '368518', '368519', '368520', '368528', '368534', '368764']
    nosy_count = 3.0
    nosy_names = ['lemburg', 'jaraco', 'tucked']
    pr_nums = ['20009', '20015']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40570'
    versions = ['Python 3.9']

    @tucked
    Copy link
    Mannequin Author

    tucked mannequin commented May 8, 2020

    518835f#diff-47c8e5750258a08a6dd9de3e9c3774acL741-R804

    That diff changed len(platform.uname()) to 5 (from 6).

    I noticed because we have some code that checks for 6 strs (arguably unnecessary),
    but I can also think of contrived examples that would break (e.g. platform.uname()[-3]).
    Interestingly, platform.uname()[5] still works.

    Was this effect intentional?

    @tucked tucked mannequin added 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels May 8, 2020
    @jaraco
    Copy link
    Member

    jaraco commented May 8, 2020

    It was intentional to address bpo-35967, although it was meant to remain compatible.

    Is len(uname()) an important behavior to retain? It feels like an implementation detail to me.

    @jaraco
    Copy link
    Member

    jaraco commented May 8, 2020

    If it is important to retain the len, it's probably also important to retain the [-N] accesses and possibly other behaviors of a length 6 tuple.

    @jaraco
    Copy link
    Member

    jaraco commented May 9, 2020

    Thanks David for the report and the draft PR (which helped me validate my thinking on the matter). In PR 20015, I've included additional tests. I've also re-written the compatibility functions to rely on the main __iter__ override.

    Another situation that's likely to be incompatible is pickleability. Is that important?

    I'm tempted to make deprecated the use of uname_result for anything other than attribute access (maybe with the ability to cast to a tuple, i.e. tuple(res)). The problem with namedtuples is that although they provide backward-compatibility for legacy behavior which returned tuples, they also bring that legacy as debt. By deprecating "access by index", that would constrain the interface to two basic usages: attribute access and iter() of all values.

    @malemburg
    Copy link
    Member

    Hi Jason,

    to achieve better backwards compatibility, it's probably better to use
    the approach taken for CodeInfo in the codecs.py module:

    class CodecInfo(tuple):
        """Codec details when looking up the codec registry"""
    
        def __new__(cls, encode, decode, streamreader=None, streamwriter=None,
            incrementalencoder=None, incrementaldecoder=None, name=None,
            *, _is_text_encoding=None):
            self = tuple.__new__(cls, (encode, decode, streamreader,
    streamwriter))
            self.name = name
            self.encode = encode
            self.decode = decode
            self.incrementalencoder = incrementalencoder
            self.incrementaldecoder = incrementaldecoder
            self.streamwriter = streamwriter
            self.streamreader = streamreader
            if _is_text_encoding is not None:
                self._is_text_encoding = _is_text_encoding
            return self
    
        def __repr__(self):
            return "<%s.%s object for encoding %s at %#x>" % \
                    (self.__class__.__module__, self.__class__.__qualname__,
                     self.name, id(self))

    This used to be a 4 entry tuple and was extended to hold additional
    fields. To the outside, it still looks like a 4-tuple in all aspects,
    but attribute access permits accessing the additional fields.

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Experts (#1, May 09 2020)
    >>> Python Projects, Coaching and Support ...    https://www.egenix.com/
    >>> Python Product Development ...        https://consulting.egenix.com/
    ________________________________________________________________________

    ::: We implement business ideas - efficiently in both time and costs :::

    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    https://www.egenix.com/company/contact/
    https://www.malemburg.com/

    @jaraco
    Copy link
    Member

    jaraco commented May 9, 2020

    Thanks Marc-Andre for the suggestion, but I don't think that approach is viable here. The whole point of bpo-35967 was to defer the execution of the .processor behavior until it was requested. The intention is not to extend the tuple, but to shorten it (at least not to require the last element to be provided at construction time). Perhaps I'm missing something here, so feel free to provide a more concrete example of what you have in mind. Just be cautious not to violate the intention (

    cpython/Lib/platform.py

    Lines 784 to 787 in 77c6146

    A uname_result that's largely compatible with a
    simple namedtuple except that 'platform' is
    resolved late and cached to avoid calling "uname"
    except when needed.
    ).

    I created bpo-40578 to track deprecation of uname for positional access.

    @jaraco
    Copy link
    Member

    jaraco commented May 9, 2020

    New changeset 2c3d508 by Jason R. Coombs in branch 'master':
    bpo-40570: Improve compatibility of uname_result with late-bound .platform (bpo-20015)
    2c3d508

    @jaraco
    Copy link
    Member

    jaraco commented May 9, 2020

    I've gone ahead and merged PR 20015 to fix the issue, but I'm happy to revisit if a better approach is proposed.

    @jaraco jaraco closed this as completed May 9, 2020
    @jaraco jaraco closed this as completed May 9, 2020
    @malemburg
    Copy link
    Member

    Ok, let me add some more context: When I wrote the uname interface
    I was aware that calling the API will take some resources. That's
    why I added the cache. IMO, that was enough as optimization.

    Now, you added a late binding optimization for the whole uname return
    tuple to save the effort of going out to the system and figure our
    the value using separate APIs or even shell access.

    I think this would have been better implemented in the various
    uname() consumers
    (

    def system():

    and below), using a variant of the uname() API, say _uname(),
    which leaves out the processor information for those APIs which don't
    need it and only provide the late binding in processor() (which could
    then also fill in the cache value for uname().

    The uname() API would then still do the full lookup, but applications
    could then use the specialized API to query only the information
    they need.

    I don't think that deprecating standard tuple access is an option
    for the uname() return value, since it's documented to be a tuple.

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Experts (#1, May 09 2020)
    >>> Python Projects, Coaching and Support ...    https://www.egenix.com/
    >>> Python Product Development ...        https://consulting.egenix.com/
    ________________________________________________________________________

    ::: We implement business ideas - efficiently in both time and costs :::

    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    https://www.egenix.com/company/contact/
    https://www.malemburg.com/

    @jaraco
    Copy link
    Member

    jaraco commented May 9, 2020

    you added a late binding optimization for the whole uname return
    tuple to save the effort of ... shell access.

    It wasn't to save the effort and it wasn't an optimization. There was a fundamental race condition where it became impossible to implement a uname command using Python because platform.system() (or anything else that delegated to platform.uname()) would unconditionally shell out to uname, and this would happen early, before a library could intercept the behavior. See bpo-35967 for more details on the motivation and challenges.

    I think this would have been better implemented...

    This idea is interesting, but I believe it also falls prey to the same race condition if any code calls platform.uname() before a Python-based uname implementation has a chance to monkeypatch the behavior.

    May I suggest that you either revive the conversation in bpo-35967 or open a new ticket to discuss improving the implementation so that the details aren't lost in this ticket, which was specifically about compatibility issues?

    I don't think that deprecating standard tuple access is an option
    for the uname() return value, since it's documented to be a tuple.

    I'll address this concern in the ticket I opened about the deprecation.

    @malemburg
    Copy link
    Member

    Hi Jason,

    I think I have to review the whole set of changes again to understand what your motivation is/was.

    For https://bugs.python.org/issue35967 I had already stated that your use case is not special enough to make the platform.py logic more complex.

    BTW: Please don't open several different tickets for the same problem. It doesn't really help to see what is going on. I'll reopen the bpo-35967 to continue the discussion there.

    Thanks.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @dennisvang
    Copy link

    @malemburg Not sure if this should be a new issue, but platform.uname docs for 3.12 still say:

    ... Returns a namedtuple() containing six attributes: system, node, release, version, machine, and processor.

    (my emphasis)

    I've got python 3.10.6 installed, which returns 5 attributes, not including processor, as expected from the discussion above:

    uname_result(
        system='Linux', 
        node='MyName', 
        release='5.15.0-48-generic', 
        version='#54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022', 
        machine='x86_64'
    )
    

    @jaraco
    Copy link
    Member

    jaraco commented Oct 6, 2022

    Moved to a new issue.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants