#1322 has deprecated platform.linux_distribution(). The feature is going to be removed without replacement functions in the stdlib. It's no longer possible to detect the Linux distribution from stdlib. Let's add a function to read /etc/os-release [1] instead. It's a very simple format and pretty standard for many years.

[1] http://0pointer.de/blog/projects/os-release

and for the older distributions without Systemd ? What do you suggest ?

Also there is an external project now aiming to provide this functionality:

https://github.com/nir0s/distro

I'm not suggesting a generic platform detection function but rather a limited function just for os-release. On platforms without /etc/os-release the function should obviously fail and raise an exception. 
/etc/os-release is available on distributions without systemd. Even my oldest Debian installation on a Raspberry Pi has it.

A large majority of Linux distributions has the file. It's available on Debian, Fedora, RHEL 6+, Ubuntu 14.04+ and many more.

-1

This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions? Who would even use that? And for what would you use it?  You have a distro module available now from PyPi, you can use that one instead, and it gives 100% correct results for all currently known distros, and can be updated independent of Python releases.

If it's that a simple function, then please implement it for your own needs.  I think it's bad style too, to rely on specific distro and release information instead of checking release names and versions.  Such a style should not be encouraged in the standard lib from my point of view.

Can you elaborate the expected API? /etc/os-release is not a single line, but a list of variables. Example on my Fedora 24:
---
$ cat /etc/os-release 
NAME=Fedora
VERSION="24 (Workstation Edition)"
ID=fedora
VERSION_ID=24
PRETTY_NAME="Fedora 24 (Workstation Edition)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:24"
HOME_URL="https://fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=24
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=24
PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy
VARIANT="Workstation Edition"
VARIANT_ID=workstation
---

I think that you should return a dictionary key=>value. Maybe we can simply return an empty dictionary if the file doesn't exist or cannot be read (but raise an error on parsing error).

FYI On Fedora 24, /etc/os-release is in fact a symbolic link to a symbolic link:
---
$ ls -l /etc/os-release 
lrwxrwxrwx. 1 root root 21 24 juin  03:25 /etc/os-release -> ../usr/lib/os-release

$ ls -l /usr/lib/os-release
lrwxrwxrwx. 1 root root 37 13 sept. 09:51 /usr/lib/os-release -> ./os.release.d/os-release-workstation

$ ls -l /usr/lib/os.release.d/os-release-workstation
-rw-r--r--. 1 root root 518 24 juin  03:25 /usr/lib/os.release.d/os-release-workstation
---

See also systemd manual page:
https://www.freedesktop.org/software/systemd/man/os-release.html

"This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions?"

According to http://0pointer.de/blog/projects/os-release systemd requires /etc/os-release. Since all major Linux distribution now use systemd, it makes sense to me to add such function.

We can document that the feature is specific to Linux (or don't declare the function on Linux?), document that it only works on recent versions of Linux distributions, and document that it doesn't work on all Linux distributions.

Please read the title of this issue. It is "Add platform.linux_os_release()". The name of the function clearly indicats that the function only works on Linux with a os-release file.

"Since all major Linux distribution now use systemd, it makes sense to me to add such function."

I'm not aware of any embedded Linux distro using systemd (no, I don't consider Raspian an embedded Linux distro).

"The name of the function clearly indicats that the function only works on Linux with a os-release file."

No, it does not. That would be linux_distribution_with_os_release_file().

I hate systemd.  So does the company I'm currently doing most of my work for.  We're using centos6 because it doesn't have systemd.  I personally use gentoo without systemd.  

That said, I don't use the platform module, so this doesn't actually affect me.

ETOOMUCHBIKESHEDDING and another uncalled systemd flame wars.

We need a fallback in the case where /etc/os-release does not exists, or just use hasattr on `platform`

if hasattr(platform, 'linux_os_release'):
   print(platform.linux_os_release())
else:
   ...

or just raise an exception if platform.linux_os_release is not available.

Sorry, it wasn't my intent to (re)start a flame war, just to point out that there are linux platforms in wide use that do not support systemd.  But I don't have an opinion on whether or not adding this function is a good idea or not.

Since we need distribution information in tests and code, I decided to reopen the bug.

I named the new function freedesktop_osrelease because it's technically not restricted to Linux. It's freedesktop.org standard that may be used by non-Linux platforms, too.

Note: os-release is **not** tight to systemd. It just happens that the standard was defined in the systemd space of freedesktop.org. I'm not aware of any major Linux platform that does not ship the file in its base os.

it's easy to say "we need it". It took a while to remove that, so I don't think it should be easily re-introduced.

so why do you need it, and why should it be exposed as part of the standard library?

What do you mean by "that"? Python never had any code to parse and handle freekdesktop.org's os-release file. Are you referring to linux_distribution() function? Petr removed platform.linux_distribution() in bpo-28167 because the function was problematic on so many levels and relied on ill defined behavior.

The os-release file has been a well-defined standard for over 8 years. These days it's available in the base image of all major Linux distributions and countless other distributions. os-release became *the* standard to identify the version and vendor of a Linux distribution. In my opinion that's reason enough to include the feature.

Matthias Klose:
> I'm not aware of any embedded Linux distro using systemd (no, I don't consider Raspian an embedded Linux distro).

As soon as the function is documented to return an error if the file doesn't exist, I don't see how this is a blocker issue.

Matthias: I don't understand your arguments against adding a Python function to read a standardize configuration file.

The platform module is the right place to add such function.

The distro module exists on PyPI because there is a need for such a function. The function even existed previously in platform. It only had to be removed because it was a pain in the *** to maintain it. The stdlib is slow to be updated, whereas Linux vendors were creative for the filename and the format of the filename.

Since the filename is now fixed and the format is well specified, IMO it's perfectly fine to add PR 23492.

> Since the filename is now fixed and the format is well specified,
> IMO it's perfectly fine to add PR 23492.

I disagree with that. The distro module is the preferred way to do this (which cannot be used in the Python core).  The rationale given is an not yet investigated issue with a timeout in tk/ttk, which from my point of view is not good enough to add a public API.

The Chef Software seems to maintain a copy of the os-release of major Linux distribution:
https://github.com/chef/os_release

Copy of the project README:

# os_release

This repo contains the /etc/os-release file from Linux distros.

## About os-release

/etc/os-release is a standard Linux file used to identify the OS, os release, and similar distros. It's now a standard file in systemd distros, but it can be found in just about every Linux distro released over the last 3-4 years.

## Why collect them

The fields in /etc/os-release are standardized, but the values are not. The only way to know that Redhat Enterprise Linux is 'rhel' or that openSUSE 15 is 'opensuse-leap' is to install the distros and check the file. This repo is a large collection of os-release files so you don't need to install each and every distro.

"The os-release file has been a well-defined standard for over 8 years."

... doesn't implicate that it's a good style to base your checks on that information.

the Chef repo seems to be a little bit out-of-date, but anyway. I don't see the relevance for this issue.

Article about os-release:
https://opensource.com/article/18/6/linux-version

"How to Check Linux Version" article:
https://linuxize.com/post/how-to-check-linux-version/

* lsb_release command <= is it still used nowadays?
* /etc/os-release file <= this PR
* /etc/issue file
* hostnamectl command
* /etc/*release file <= covered by the PyPI distro module
* uname command <= platform.uname()

The os-release file is not tight to systemd. Only the reverse relationship is true: systemd, d-bus and other software require os-release.

The file is present in the minimal base image of distributions like Alpine, ArchLinux, CentOS, Debian, Fedora, RHEL, SUSE, and Ubuntu plus all derivates. In the last three years I have not seen any Linux distribution that is missing the file. The only odd-ball in the list is Alpine. It ships /usr/lib/os-release without the optional but recommended /etc/os-release symlink.

If you know any Linux platform without os-release, please let me know. I'm curious to know which platforms don't have the file and why the platform excludes it.

Python could also follow the lead of other software like D-Bus and make the presence of os-release mandatory on Linux. Any Linux platform without it would be considered broken. I don't think it is necessary to impose such restriction on vendors.

> * /etc/os-release file <= this PR
> * /etc/*release file <= covered by the PyPI distro module

no Victor, you are wrong.  os-release is covered by the distro module. There's no need to expose a second implementation as another API in the standard library.

Matthias Klose
> "The os-release file has been a well-defined standard for over 8 years."
> ... doesn't implicate that it's a good style to base your checks on that information.

IMO /etc/os-release is a reliable way to retrieve the Linux distribution name. What do you suggest to get the Linux distribution otherwise?


> the Chef repo seems to be a little bit out-of-date, but anyway. I don't see the relevance for this issue.

Someone can use it to test the implementation.

IMO it's interesting to see which Linux distributions provide os-release. The list of quite long!

* alpine_3_8
* amazon_2
* amazon_2018
* antergos
* arch
* archarm
* centos_7
* clearlinux_1
* clearos_7
* cumulus_3_7
* debian_7
* debian_8
* debian_9
* elementary_5
* fedora_28
* gentoo
* ios_xr_6
* kali_2018_4
* linuxmint_18_2
* linuxmint_19
* mangeia_6
* manjaro
* nexus_7
* nixos
* opensuseleap_15_0
* opensuseleap_15_1
* opensuseleap_42_3
* oracle_7
* rancheros_1_4
* raspbian_10
* raspbian_8
* redhat_7
* redhat_8
* scientific_7
* slackware_14_2
* sled_12_3
* sled_15
* sles_11_4
* sles_12_3
* sles_15_0
* sles_15_1
* sles_sap_12_0
* sles_sap_12_1
* sles_sap_12_2
* sles_sap_12_3
* ubuntu_1404
* ubuntu_1604
* ubuntu_1804
* xbian
* xcp_7_5_0
* xenserver_7_6

It's not like no Linux distribution provide the file. I don't know most of these Linux distributions :-D

Christian, is it necessary to start with threats?

> Python could also follow the lead of other software like D-Bus
> and make the presence of os-release mandatory on Linux.
> Any Linux platform without it would be considered broken.
> I don't think it is necessary to impose such restriction on vendors.

Python *does* follow this lead with the distro module. And no, I don't see yet any argument why you need a second API for that.

> os-release is covered by the distro module.

Ah, I didn't know.

https://pypi.org/project/distro/

"""
The distro package implements a robust and inclusive way of retrieving the information about a distribution based on new standards and old methods, namely from these data sources (from high to low precedence):

    The os-release file /etc/os-release, if present.
    The output of the lsb_release command, if available.
    The distro release file (/etc/*(-|_)(release|version)), if present.
    The uname command for BSD based distrubtions.
"""

You're right, not only os-release is used, but it has the highest priority.


> There's no need to expose a second implementation as another API in the standard library.

Usually, a new experimental feature is developed on PyPI. Once it becomes stable and mature enough, we consider to include it into the stdlib.

Since the file format became standard and the file became available on almost all Linux distributions, it seems like we reached this point, no?


Christian: can you please try to contact the distro maintainer, Nir Cohen, to ask him to help us on this issue?

> Python could also follow the lead of other software like D-Bus and make the presence of os-release mandatory on Linux. Any Linux platform without it would be considered broken. I don't think it is necessary to impose such restriction on vendors.

I don't see the point. Also, your function raises an error if the file is missing. That's fine. There is no need to go further.

-1

android and aosp, termux are also linux distributions without /etc/os-release

the only thing reliable there is probably sysconfig.get_config_vars('MULTIARCH') when properly used.

Matthias Klose:
> There's no need to expose a second implementation as another API in the standard library.

When linux_distribution() has been removed, platform.platform() became less useful. Example:

$ python2 -m platform
Linux-5.9.8-200.fc33.x86_64-x86_64-with-fedora-33-Thirty_Three

$ python3 -m platform
Linux-5.9.8-200.fc33.x86_64-x86_64-with-glibc2.32

Python 2 provides *more* information than Python 3. I see this as a regression.

It would be great if we could again add again the Linux distribution name and vresion in platform.platform().

For example, I'm using platform.platform() on buildbots to quickly identify on which operating systems an issue happens: see test.pythoninfo script ("make pythoninfo" used by buildbots).

Right now, it's annoying to have to manually dig into the buildbot worker details to get informations, usually outdated (when the machine is upgraded, but not the worker description).

pmp-p:
> android and aosp, termux are also linux distributions without /etc/os-release

I replied ealier. It's fine that platform.freedesktop_os_release() (I prefer a name with two underscores :-)) raises an error on embedded Linux distributions. Usually, when you work on embedded devices, you fully control your toolkit and what is shipped on it.

For Android, there is already https://docs.python.org/dev/library/sys.html#sys.getandroidapilevel function.

Open question: is Android a Linux distribution? ;-)


> the only thing reliable there is probably sysconfig.get_config_vars('MULTIARCH') when properly used.

It looks unrelated. It doesn't provide the Linux distribution name or version.

> IMO it's interesting to see which Linux distributions provide os-release. The list of quite long!

As I said before I could not find any supported release of a Linux distribution without a proper os-release file. It might be possible that there are micro container images or Linux From Scratch builds without os-release. So far I haven't found any distribution and I have tested almost two dozen.

> Christian, is it necessary to start with threats?

I'm sorry that you feel threatened. It was not my intention to make you feel that way.

IMHO the Python core team can define requirements for vendors and distributors. PEP 11 an PEP 394 contain examples of recommendations and requirements. After all it's in our interest to ensure that our users get a good and consistent experience on supported platforms. Of course vendors are free to ignore our requirements and recommendations.

In the case of this it's mood to even discuss requiring os-release. (a) the PR does not add a hard dependency on os-release, and (b) the file is present anyway.

> In the case of this it's mood to even discuss requiring
> os-release. (a) the PR does not add a hard dependency
> on os-release, and (b) the file is present anyway.

For anything outside cpython, there is the distro module, and the distro module is checking os-release first if it exists.  I don't see a good value to add a second API for a subset of the distro module.  Apparently the distro module is used by other projects.

$ apt rdepends python3-distro | grep '^  [DRS]' | wc -l
33

If you think, distro information is needed to run cpython's tests, then make that function available as part of the tests, and don't promote that as a new API, although it is better in many cases to rely on feature tests instead of version information and distro names.  Don't expose it as yet another API.

Correction: It was ArchLinux, not Alpine. ArchLinux used to have /usr/lib/os-release only. Recent ArchLinux releases has a symlink from /etc/os-release to /usr/lib/os-release.

The distro module has been mentioned multiple times. I looked at its code base and I suggest to not add it to  the Python stdlib. It contains code specific to some Linux distributions. Examples.

elif 'ubuntu_codename' in props:
    # Same as above but a non-standard field name used on older Ubuntus
    props['codename'] = props['ubuntu_codename']

NORMALIZED_LSB_ID = {
    'enterpriseenterpriseas': 'oracle',  # Oracle Enterprise Linux 4
    'enterpriseenterpriseserver': 'oracle',  # Oracle Linux 5
    'redhatenterpriseworkstation': 'rhel',  # RHEL 6, 7 Workstation
    'redhatenterpriseserver': 'rhel',  # RHEL 6, 7 Server
    'redhatenterprisecomputenode': 'rhel',  # RHEL 6 ComputeNode
}

basenames = ['SuSE-release',
             'arch-release',
             'base-release',
             'centos-release',
             'fedora-release',
             'gentoo-release',
             'mageia-release',
             'mandrake-release',
             'mandriva-release',
             'mandrivalinux-release',
             'manjaro-release',
             'oracle-release',
             'redhat-release',
             'sl-release',
             'slackware-version']

platform.linux_distribution() has been removed because it was difficult to keep the implementation up to date, whereas Python are rarely or not updated during the lifecycle of a Linux distribution version.

--

The os-release file is different: the filename is standardized and the file format is standardized. I expect really minor maintenance on a function parsing it.

Note: distro doesn't specify an encoding when opening os-release, but use the locale encoding. I suggest to use UTF-8.

The distro module remains useful since it tries to better API. For example, variable names are converted to lowercase and it extracts the codebase from the version variable:

        elif 'version' in props:
            # If there is no version_codename, parse it from the version
            codename = re.search(r'(\(\D+\))|,(\s+)?\D+', props['version'])
            if codename:
                codename = codename.group()
                codename = codename.strip('()')
                codename = codename.strip(',')
                codename = codename.strip()
                # codename appears within paranthese.
                props['codename'] = codename

But again, I don't think that we should implement such heuristics (code specific to some Linux distributions) in the stdlib, to minimize the maintenance burden.

From PR discussion on GH:

I made ID_LIKE a special case because it's likely a users will use the field in a wrong way. The issue won't be detected in common CI because the field is either not present or contains a single item for majority of Linux distros. Common values are debian or fedora. For example Ubuntu has ID_LIKE=debian. Only very few distros have an ID_LIKE with multiple entries.

Victor asked me to remove the special case because splitting is trivial. In that case users would have to split the field and always check for "name in info["ID_LIKE"].split()".

The special case makes it less likely that users get the field wrong and reduces their burden.

I'm -1 on adding any kind of replacement for platform.linux_distribution() tp Python's stdlib. The experiment has failed and we should acknowledge this.

The main reason why it failed was the Linux distros keep inventing new ways to identify themselves, sometimes misuse existing mechanisms to maintain compatibility (e.g. as a Ubuntu or RedHat based OS) or ship with two different release files.

At the time we removed the API, we said that people should use PyPI package to get this information, since those can much more easily be maintained with the ever changing patterns distributions use or invent.

The distro package is one package and if it doesn't satisfy the needs, simply create a new one which does.

Note that the original implementation had a mechanism to read such release files. Even the names of those files changed every few years. People appear to have settled on "os-release" nowadays, since using "<distro>-release" was found to create too much confusion, but this is not fixed and when it comes to parsing the contents, I'm pretty sure that there a subtle differences in how to interpret them between distros.

Then the LSB standardized this and wanted everyone to use lsb_release. Guess what... The tool is not even installed anymore by default on recent OpenSUSE releases. Experience shows that these things change too often to make the stdlib a good place to maintain support for this.

It's not a replacement for platform.linux_distribution().

On 25.11.2020 10:06, Christian Heimes wrote:
> 
> It's not a replacement for platform.linux_distribution().

Right, not even that :-)

It would be an interface to a file /etc/os-release
that's common nowadays, just like /etc/lsb-release was some years
ago. These things change too often to make the stdlib a good fit.
I'm pretty sure distros will invent something new in 5 years which
would then render the API mostly useless again.

To get a sense of the complexity involved in all this, have a look
at how Ansible does this:

https://github.com/ansible/ansible/blob/ea119d30894478b84b5fbe271f580cb2b0401b86/lib/ansible/module_utils/facts/system/distribution.py

It's not just about the file name, the content also needs a lot
of massaging to make it useful for a general purpose API. And
the content quirks are not set in stone either. Here's a ticket
where Debian, for example, drops the version info from the file:

https://github.com/ansible/ansible/issues/19874

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               https://www.egenix.com/company/contact/
                     https://www.malemburg.com/

> It would be an interface to a file /etc/os-release
> that's common nowadays, just like /etc/lsb-release was some years
> ago. These things change too often to make the stdlib a good fit.
> I'm pretty sure distros will invent something new in 5 years which
> would then render the API mostly useless again.

That's pointless speculation -- unless you found a fool-proof way to see into the future.

Where do you draw the line? Should the stdlib ignore any standard and API that is less than 10 years old? 20 years?

> Where do you draw the line?

there's a module available outside cpython. Your use case is to add some work around for some tests (like issue42142).  If you need it for the tests, add it to the test framework.

On 25.11.2020 10:39, Christian Heimes wrote:
> 
>> It would be an interface to a file /etc/os-release
>> that's common nowadays, just like /etc/lsb-release was some years
>> ago. These things change too often to make the stdlib a good fit.
>> I'm pretty sure distros will invent something new in 5 years which
>> would then render the API mostly useless again.
> 
> That's pointless speculation -- unless you found a fool-proof way to see into the future.
>
> Where do you draw the line? Should the stdlib ignore any standard and API that is less than 10 years old? 20 years?

It's not pointless. The rate of change in the field is why this
particular API did not work out in practice. It was working fine
at the time I added it, but then quickly became unmaintainable.

Note that I'm not saying it's a useless API. The main point is
that such code is better placed into a PyPI package, where it
can be updated more frequently and also in a safe way across
Python versions.

Additionally, such a package could provide normalization features,
to make matching distros easier, even when the distros decide
to change the way they are named, e.g. for marketing purposes.

The distro package is doing a pretty good job at this.

If you need this for tests in the stdlib, I'd suggest to add a helper
to the test package to find the needed information for the platforms
where you want to run platform dependent tests.

For tests outside the stdlib, the distro package will do the
job just fine.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               https://www.egenix.com/company/contact/
                     https://www.malemburg.com/

Marc-Andre Lemburg:
> The main reason why it failed was the Linux distros keep inventing new ways to identify themselves, sometimes misuse existing mechanisms to maintain compatibility (e.g. as a Ubuntu or RedHat based OS) or ship with two different release files.
> (...)
> Then the LSB standardized this and wanted everyone to use lsb_release. Guess what... The tool is not even installed anymore by default on recent OpenSUSE releases. Experience shows that these things change too often to make the stdlib a good place to maintain support for this.

Please read the discussion. os-release is a standardize file: fixed filename, only 2 possible locations, standardize format. It's a freedesktop specification. In my experience, freedesktop standards are well adopted, whereas LSB standards look like a failed attempt.

This is no need to bet on the future. os-release file is already available on all major Linux distribution, as, again, explained in depth in previous comments. There is no need to install a lsb_release program or anything. It's part of the *base* image on these systems.

As you wrote, lsb_release isn't installed on many operating systems. For example, I just checked on my Fedora 33, it's not installed... Moreover, I'm not comfortable with running an external program which causes its own set of issue. Reading a plain text file is safer and faster.


> To get a sense of the complexity involved in all this, have a look
at how Ansible does this:
> https://github.com/ansible/ansible/blob/ea119d30894478b84b5fbe271f580cb2b0401b86/lib/ansible/module_utils/facts/system/distribution.py

Ansible has a high-level API similar to what the Python distro module does: it calls uname, parse /etc/*release files, try to normalize values, has special cases, etc.

Not everybody needs like kind of advanced API. For my own use cases (see previous comments), I'm perfectly fine with basic and incomplete information.


> It's not just about the file name, the content also needs a lot
> of massaging to make it useful for a general purpose API. (...)

I strongly suggest to keep platform.freedesktop_os_release() as simple as possible: just parse the file, no special case at all.

For example, I'm against treating ID_LIKE variable differently in PR 23492. Users would expect if another space separated variable is added tomorrow, the function would also specialize it, and so we would come back to the linux_distribution() maintenance issue.

IHMO a dummy text parser remains very useful. The file format is not as trivial as it looks, there are single quote and double quotes, and escaped characters.

If someone wants a more complete API, I suggest to develop it on PyPI on top of the new function.

It's fine to provide any API where some variables can miss if the there are not provided by the operating system. For me, it's like the os and select modules which are thin wrappers to operating system functions (syscalls). And then there are more high-level module like shutil and selectors. But we don't need to ship a high-level API for os-release.

Platform was always a thin wrapper to OS functions. For example, there is no unified API to retrieve OS name and version on Windows, macOS or Linux. You need to pick the proper function. For me, freedesktop_os_release() just follows this trend.

Marc-Andre Lemburg:
> It's not pointless. The rate of change in the field is why this
> particular API did not work out in practice. It was working fine
> at the time I added it, but then quickly became unmaintainable.

Please look at the history of the os-release file. The os-release file format exists for 8 years and has not changed in 8 years.

It seems like you are referring to /etc/*release files. I agree that these files are causing a lot of maintenance troubles, and... that's exactly why os-release file has been standardized and created.

So I don't get your point.

On 25.11.2020 11:13, STINNER Victor wrote:
> Platform was always a thin wrapper to OS functions. For example, there is no unified API to retrieve OS name and version on Windows, macOS or Linux. You need to pick the proper function. For me, freedesktop_os_release() just follows this trend.

Not really. We have functions per OS, but not functions which only work
on a subset of distros of an OS.

The patch also has other issues:

A text file parse could be a private function in the module,
but it doesn't fit the platform module API spirit.

platform module APIs should return meaningful information and
provide defaults where these cannot be determined. Accordingly,
an API would have to return a tuple (distname, version, id), just
like linux_distribution() did.

Regardless, I don't see the point of opening up this can of
worms again. We settled on moving Linux distribution version detection
out of the stdlib and that was a good decision.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               https://www.egenix.com/company/contact/
                     https://www.malemburg.com/

> It's not pointless. The rate of change in the field is why this
> particular API did not work out in practice. It was working fine
> at the time I added it, but then quickly became unmaintainable.

Yes, it's pointless. You are arguing that os-release is bad because lsb-release was bad. It's like arguing against TOML file format because YAML and JSON have issues.

os-release was purposely designed to address concerns with lsb-release. os-release has been around for more than a decade and wildly adopted (except on Android platforms). The format hasn't changed since its first appearance in November 2010. Some optional fields were standardized and the meaning of "no shell features are supported" was elaborated on.

PS: MAL, would it be possible to suppress your email footer? BPO is not an advertisement channel.

On 25.11.2020 11:45, Christian Heimes wrote:
> 
> Christian Heimes <lists@cheimes.de> added the comment:
> 
> PS: MAL, would it be possible to suppress your email footer? BPO is not an advertisement channel.

That's an ancient bug on bpo, which apparently still haven't been
resolved - it doesn't handle longer sigs correctly. Sorry for that.

Pablo, I would like to get mediation from the release manager.

I want to add an interface to the os-release file. It's a decade-old standard from freedesktop.org. The file is available in the minimal base installation of virtually every Linux distributions except for Android. It's in Alpine, Arch Linux, Debian family (Mint, Raspberry Pi OS, Ubuntu, ...), Fedora family (CentOS, RHEL, ...), Gentoo, SUSE, and many more. The os-release file contains human readable and machine readable operating system information. The file format is simple but not trivial.

Matthias and Marc-Andre oppose my patch. Victor and I are in favor of the new feature.

I just needed such functionality for PTY tests: https://github.com/python/cpython/pull/23514

> I just needed such functionality for PTY tests

another use case for a test. so please add it to the test framework, also suggested by Marc.

Don't get me wrong.
I don't want to put my vote in this hot discussion but mention my use case. 
test.support can solve my need.
On another hand, very many Python batteries exist to solve third-party needs and are not required by CPython itself.

So I'm +0 for adding this function -- but can live without it.

We have reached an impasse. Therefore I have contacted the steering council and requested mediation. Victor will abstain from the decision process.

This issue was brought to the Python Steering Council, and we deliberated it at today's SC meeting.  With a vote of 4 approvals and one abstention, we have approved the addition of this API.

I don't agree with this decision, but ok.

In this case, please also provide a way to provide the value of the VERSION_ID field. It doesn't make sense to blacklist whole distributions in tests.

Matthias Klose:
> It doesn't make sense to blacklist whole distributions in tests.

Adding freedesktop_os_release() and using it are two different topics. I understand the the SC decision is on adding the function.

Using it to skip tests should be decided on a case by case basic. In general, I really dislike to skip tests on a specific Linux distribution.

In bpo-41818, the os-release file was used as a temporary fix just to unblock buildbots, but it was quickly removed. It was not used to skip a test on Gentoo, but to run the test on Gentoo ;-) Also, it was quickly removed, and it seems like even the skip on BSD was removed. Great! It was there for a very long time.

> In this case, please also provide a way to provide the value of the VERSION_ID field.

I'm not sure what you mean. PR 23492 exposes the whole content of the os-release file as a dict. There is no special case for any field.

VERSION_ID can be retrieved as any field with: platform.freedesktop_os_release().get('VERSION_ID', '').

Example on Fedora 33.

$ cat /etc/os-release 
NAME=Fedora
VERSION="33 (Workstation Edition)"
ID=fedora
VERSION_ID=33
VERSION_CODENAME=""
PLATFORM_ID="platform:f33"
PRETTY_NAME="Fedora 33 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:33"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=33
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=33
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation


$ ./python 
Python 3.10.0a2+ (heads/pr/23492:6a4c92bcbe, Nov 30 2020, 22:29:23) 
[GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform, pprint
>>> os_release=platform.freedesktop_os_release()

>>> pprint.pprint(os_release)
{'ANSI_COLOR': '0;38;2;60;110;180',
 'BUG_REPORT_URL': 'https://bugzilla.redhat.com/',
 'CPE_NAME': 'cpe:/o:fedoraproject:fedora:33',
 'DOCUMENTATION_URL': 'https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/',
 'HOME_URL': 'https://fedoraproject.org/',
 'ID': 'fedora',
 'LOGO': 'fedora-logo-icon',
 'NAME': 'Fedora',
 'PLATFORM_ID': 'platform:f33',
 'PRETTY_NAME': 'Fedora 33 (Workstation Edition)',
 'PRIVACY_POLICY_URL': 'https://fedoraproject.org/wiki/Legal:PrivacyPolicy',
 'REDHAT_BUGZILLA_PRODUCT': 'Fedora',
 'REDHAT_BUGZILLA_PRODUCT_VERSION': '33',
 'REDHAT_SUPPORT_PRODUCT': 'Fedora',
 'REDHAT_SUPPORT_PRODUCT_VERSION': '33',
 'SUPPORT_URL': 'https://fedoraproject.org/wiki/Communicating_and_getting_help',
 'VARIANT': 'Workstation Edition',
 'VARIANT_ID': 'workstation',
 'VERSION': '33 (Workstation Edition)',
 'VERSION_CODENAME': '',
 'VERSION_ID': '33'}

>>> os_release['VERSION_ID']
'33'

New changeset 5c73afc36ee6cca41009a510092e1f901c5dc0a0 by Christian Heimes in branch 'master':
bpo-28468: Add platform.freedesktop_os_release() (GH-23492)
https://github.com/python/cpython/commit/5c73afc36ee6cca41009a510092e1f901c5dc0a0

I merged Christian's PR 23492. The initial issue, add a function to parse the os-release file, is now solved, so I close the issue.

If you want to use it in a test, please open a new issue.

Thanks everyone for this constructive discussion, IMO the final merged function is now better thanks to that. And thanks Christian for pushing the feature and implementing it ;-)

New changeset bfda4f5776fd20f92b441c8a88ee7a9d44dc8777 by Victor Stinner in branch 'master':
bpo-28468: Fix typo in _os_release_candidates (GH-23913)
https://github.com/python/cpython/commit/bfda4f5776fd20f92b441c8a88ee7a9d44dc8777