Issue42173
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-10-27 15:59 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
Oracle_Solaris_full_test.txt | kulikjak, 2020-11-02 08:18 | |||
Oracle_Solaris_detailed_test.txt | kulikjak, 2020-11-02 08:18 |
Pull Requests | |||
---|---|---|---|
URL | Status | Linked | Edit |
PR 23002 | closed | vstinner, 2020-10-27 16:02 |
Messages (49) | |||
---|---|---|---|
msg379767 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-27 15:59 | |
In past years, very few Python core developers took care of the Solaris support. The latest significant Solaris enhancement was in 2018 with the addition of a new Solaris st_fstype attribute to os.stat(), in Python 3.7, by Jesús Cea Avión (bpo-32659). Another recent Solaris-related change is bpo-41687: "Fix implementation of sendfile to be compatible with Solaris". On the Oracle side, there was an article about Python on Solaris in 2019: "Future of Python on Solaris" by Darren Moffat https://blogs.oracle.com/solaris/future-of-python-on-solaris I now that Giampaolo Rodola cares of the Solaris support for his psutil project: https://github.com/giampaolo/psutil/ It's unclear to me if Oracle still plans to maintain Solaris. The latest release was published 2 years ago. The Wikipedia article says: "While Oracle did have a large layoff of Solaris development engineering staff, development continues today of which Solaris 11.4 was released in 2018." https://en.wikipedia.org/wiki/Solaris_(operating_system) There is currently 25 open issues which could be closed if we drop the Solaris support. See also the PEP 11 for the process. I asked asked 4 years ago (in 2016) if Solaris support should/can be removed: "OpenIndiana and Solaris support" https://mail.python.org/archives/list/python-dev@python.org/message/NOT2RORSNX72ZLUHK2UUGBD4GTPNKBUS/ Python has no longer buildbot workers running on Solaris or a Solaris variant (OpenIndiana, Illumos, etc.). I propose to drop the Solaris support in Python to reduce the maintenance burden. Python has a few features specific to Solaris: * st_fstype attribute of os.stat() * select.devpoll and selectors.DevpollSelector * time.CLOCK_HIGHRES constant * stat.S_ISDOOR() * tarfile: "Solaris extended header" (no need to remove this one) Solaris has many names and variants: * SunOS * SRV4 * Solaris * Illumos (still active in July 2020 at least) * OpenSolaris: "On Friday, August 13, 2010, details started to emerge relating to the discontinuation of the OpenSolaris project and the pending release of a new closed-source, proprietary version of Solaris, Solaris 11." says Wikipedia * OpenIndiana (latest release in 2020) I'm not sure on how to process? Deprecate all Solaris-specific features in Python 3.10, make sure that Python 3.11 cannot be built on Solaris anymore, and drop the code in Python 3.12? Or drop everything as soon as possible in Python 3.10? The alternative is to leave the code unchanged and let the code slowly die, as we did for other platforms like HP-UX and IRIX. See also my notes on platforms supported by Python: https://pythondev.readthedocs.io/platforms.html Attached draft PR gives an idea on how much code can be removed if Solaris support is removed. A coase estimation is the removal of 700 lines: Doc/c-api/init.rst | 3 +- Doc/distutils/apiref.rst | 1 - Doc/library/gettext.rst | 10 +- Doc/library/os.rst | 12 +- Doc/library/platform.rst | 5 +- Doc/library/posix.rst | 8 +- Doc/library/sys.rst | 3 +- Doc/library/sysconfig.rst | 1 - Doc/library/time.rst | 13 +- Doc/using/unix.rst | 7 - Include/pyport.h | 5 - Lib/cgi.py | 4 +- Lib/ctypes/test/test_byteswap.py | 2 +- Lib/ctypes/util.py | 100 ++-------- Lib/distutils/command/bdist.py | 2 +- Lib/distutils/tests/support.py | 2 +- Lib/distutils/tests/test_build_ext.py | 21 -- Lib/distutils/util.py | 11 -- Lib/gettext.py | 3 - Lib/logging/handlers.py | 2 - Lib/platform.py | 27 +-- Lib/sysconfig.py | 11 -- Lib/test/fork_wait.py | 4 +- Lib/test/subprocessdata/fd_status.py | 3 - Lib/test/test__locale.py | 24 --- Lib/test/test_asyncio/test_sendfile.py | 6 - Lib/test/test_asyncore.py | 2 - Lib/test/test_cmd_line.py | 2 +- Lib/test/test_curses.py | 2 +- Lib/test/test_fileio.py | 2 +- Lib/test/test_importlib/import_/test_path.py | 2 +- Lib/test/test_locale.py | 2 +- Lib/test/test_os.py | 14 +- Lib/test/test_posix.py | 7 +- Lib/test/test_pty.py | 4 - Lib/test/test_selectors.py | 2 +- Lib/test/test_shutil.py | 2 +- Lib/test/test_socket.py | 4 - Lib/test/test_strftime.py | 3 +- Lib/test/test_sys.py | 2 +- Lib/uuid.py | 5 - Modules/Setup | 4 - Modules/_ctypes/ctypes.h | 4 - Modules/_cursesmodule.c | 2 +- Modules/_multiprocessing/multiprocessing.h | 15 -- Modules/_posixsubprocess.c | 11 -- Modules/errnomodule.c | 23 +-- Modules/mathmodule.c | 2 +- Modules/posixmodule.c | 65 +----- Modules/signalmodule.c | 2 +- Modules/socketmodule.c | 7 - Modules/termios.c | 8 +- Modules/timemodule.c | 9 +- Python/bootstrap_hash.c | 17 +- Python/fileutils.c | 3 +- Python/import.c | 3 +- Tools/freeze/README | 5 - Tools/i18n/pygettext.py | 3 +- Tools/pynche/Main.py | 2 - Tools/pynche/README | 2 +- Tools/scripts/nm2def.py | 2 +- configure | 285 +-------------------------- configure.ac | 101 +--------- pyconfig.h.in | 19 +- setup.py | 20 +- 65 files changed, 91 insertions(+), 868 deletions(-) |
|||
msg379886 - (view) | Author: Sebastian Wiedenroth (wiedi) * | Date: 2020-10-29 22:13 | |
Please don't. Even if Oracle doesn't care anymore, the open source illumos community does. We (illumos) share the same uname "SunOS" and depend on this support. I have offered to host an illumos buildbot in the past and that offer is still valid. At that time the testsuite did not pass and I could not figure out the fix yet. If having this fixed is a requirement to keep illumos supported I will invest more time into this. I can also offer shell access to anyone interested in helping with this. |
|||
msg379887 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-29 22:13 | |
I looked at PyPI download statistics of the "six" module over the last 180 days (6 months), grouped by operating system name: * SunOS: 3,975 downloads * TOTAL: 590,727,298 downloads SunOS is 0.0007% of all six downloads. For comparison, the top 4 is: * Linux: 91% (535,804,817) * Windows: 4% (21,053,173) * <empty system name>: 3% (18,695,360) * macOS (Darwin): 3% (14,776,993) Raw data of the top 14: num_downloads,system_name 535804817,Linux 21053173,Windows 18695360, 14776993,Darwin 347090,FreeBSD 9759,Qnx 5350,Java 3975,SunOS 3178,CYGWIN_NT-10.0-14393 3177,CYGWIN_NT-10.0-17763 2505,CYGWIN_NT-10.0-18363 2263,AIX 2061,NetBSD 1891,OpenBSD |
|||
msg379888 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-29 22:14 | |
> We (illumos) share the same uname "SunOS" and depend on this support. Which Python version do you use? https://blogs.oracle.com/solaris/future-of-python-on-solaris mentions Python 2.7 and 3.4. |
|||
msg379890 - (view) | Author: Sebastian Wiedenroth (wiedi) * | Date: 2020-10-29 22:22 | |
> Which Python version do you use? Myself currently mostly python 3.8 via pkgsrc on SmartOS (one of the illumos distributions). OmniOS ships 3.7: https://github.com/omniosorg/omnios-build/tree/master/build/python37 OpenIndiana ships at least 2.7, 3.5, 3.7. The illumos build process also depends on python and was updated to support 3.9 two weeks ago: https://www.illumos.org/issues/13217 |
|||
msg379891 - (view) | Author: Ondřej Súkup (mimi.vx) | Date: 2020-10-29 22:36 | |
Oracle moved solaris 11.4 to +- rolling release mode, last SRU is 26 from 20.10.2020 with month release cadence. On another side, almost all informations are behind the Oracle paywall |
|||
msg379893 - (view) | Author: Jesús Cea Avión (jcea) * ![]() |
Date: 2020-10-29 22:38 | |
I think Solaris Support is still important and I could invest more time on it. About package downloads: 1. I routinely download packages in my laptop (Linux) and distribute them to a fleet of servers (Linux and Solaris) via ansible and similar. Many servers in intranets don't install packages directly via PIP. 2. Under OmniOS or SmartOS (Illumos distributions), many people use the system package manager to install Python packages, not "PIP". For instance, http://netbsd.2816.n7.nabble.com/WIP-commit-Upgrade-to-Mailman-2-1-34-td427426.html. Buildbots: I used to manage a buildbot with OpenIndiana, but it didn't survived the GITHUB migration. I could provide now a SmartOS buildbot. As a core developer, I do care about Solaris. I fully realize I have been quite inactive since the github migration. My fault. I could devote more time to the task. |
|||
msg379894 - (view) | Author: Jesús Cea Avión (jcea) * ![]() |
Date: 2020-10-29 22:45 | |
I am using Python 3.8 installed via pkgsrc in SmartOS with some Zones running Python 3.9 self compiled code. More about the downloads stats: 4% packages in windows: I am pretty sure that most windows programmers install packages via conda-like tools. Surely there is more that 4% of python programmers on windows, even more in machine learning and data science areas, where windows and MacOS X rule (apparently being only 7% of PIP downloads). Even if the PIP download stats would reflect penetration, Solaris is ahead of NetBSD and OpenBSD together. |
|||
msg379895 - (view) | Author: Joshua M. Clulow (jclulow) | Date: 2020-10-29 22:48 | |
Hi! I'm a member of the illumos core team, and I'm also pretty keen for us to keep Python support! Some of our core OS build and packaging tooling is written in Python, and certainly applications like Synapse (Matrix) and Review Board are important to myself. We currently provide a virtual machine for builds to the Go project that is wired in to their official CI/CD. I expect we can help out with a similar thing here! I know some illumos distributions (OmniOS, OpenIndiana, anything with pkgsrc, etc) are maintaining some patches against the stock Python to fix failing tests, etc. Hopefully we can figure out how to get those things upstreamed and increase our participation there. |
|||
msg379897 - (view) | Author: Andy Fiddaman (omnios) | Date: 2020-10-29 22:59 | |
We at OmniOS (an illumos distribution) are in the process of upgrading to Python 3.9 and working on getting tests clean. We're in pretty good shape with a few local patches and only having to skip a few tests at the moment, and most of those patches should be suitable for integrating generally. Some of the tests just need adjusting to be aware of the different platform (e.g. illumos does not support multiple SCM_RIGHTS messages in a single packet), some still need investigating to find the root cause, but the list is small. |
|||
msg379899 - (view) | Author: Andy Fiddaman (omnios) | Date: 2020-10-29 23:06 | |
Re: Downloads OmniOS makes modules like 'six' available through its own IPS packaging system (which is written in Python) so installations won't show up pypi stats. six is a core module on the platform so it is installed on 1000s of OmniOS machines (and then large numbers of SmartOS, OpenIndiana and tribblix ones too - all using the SunOS tag). The numbers aren't going to be in the same league as other platforms but the downloads stats definitely don't represent the illumos installation base and dependency on Python. |
|||
msg379900 - (view) | Author: hajma (tropikhajma@gmail.com) | Date: 2020-10-29 23:39 | |
Like OmniOS, Oracle Solaris provides the 'six' (and many other) module prepackaged. You can review a clone of the development repository at https://github.com/oracle/solaris-userland/tree/master/components/python/six |
|||
msg379901 - (view) | Author: Peter Firmstone (pfirmsto) | Date: 2020-10-30 00:47 | |
No, please continue supporting Illumos, I'm still using it. Illumos is still relevant on servers. |
|||
msg379906 - (view) | Author: brett (brett3) | Date: 2020-10-30 01:44 | |
Between SmartOS, Illumos, and mainline Solaris, there is ample interest and support for ensuring that SunOS derivatives remain 1st class python platforms. Platform genocide is both unnecessary and unwarranted. |
|||
msg379910 - (view) | Author: Robert D. French (robertdfrench) | Date: 2020-10-30 03:28 | |
Hello! I just want to lend my voice in favor of maintaining "Solarish" support as well, and offer what help I may for resolving issues. |
|||
msg379915 - (view) | Author: jason matthews (jm650) | Date: 2020-10-30 04:24 | |
Please do not drop support. Python is widely used in the "Solaris" community with most people downloading packages from Openindiana/Hipster or Joyent/SmartOS repos. Most "Solaris" users are not using Solaris proper. The download stats published in this ticket is not a good representation of usage. Similarly most (Free|Open|*)BSD users install from their respective versions of ports. |
|||
msg379924 - (view) | Author: Tony Albers (tbalbers) | Date: 2020-10-30 07:21 | |
No no no, please don't. Apart from FreeBSD, illumos distros are the only really hard-core UNIX OS'es still freely available, the features taken into account. SMF, dtrace and several hypervisor types makes illumos really stand out. I understand that there are resources that need to be assigned to maintaining Python on illumos/SunOS, but please reach out to their communities, maybe someone can help. |
|||
msg379925 - (view) | Author: Marius Pana (mariuspana) | Date: 2020-10-30 07:28 | |
Please do not drop support for SunOS. We are happy to provide some hosting services on SmartOS/Illumos for building. |
|||
msg379926 - (view) | Author: Jasper Siepkes (siepkes) | Date: 2020-10-30 07:33 | |
I'm another Illumos user that has crawled from under a rock ;-) to request not to drop Illumos support. |
|||
msg379927 - (view) | Author: Carsten Grzemba (cgrzemba) | Date: 2020-10-30 07:37 | |
Please continue support for Solaris/IllumOS! For build resources on the most recent Solaris platforms you can contact opencsw.org. |
|||
msg379933 - (view) | Author: Paolo Marcheschi (marcheschi) | Date: 2020-10-30 08:17 | |
Please continue support for Solaris/IllumOS! This is very important for us. |
|||
msg379937 - (view) | Author: abarbu (abarbu) | Date: 2020-10-30 08:44 | |
Please don't drop Solaris support, we still use it to this day. |
|||
msg379938 - (view) | Author: Predrag Zečević (predrag.zecevic.1961) | Date: 2020-10-30 08:46 | |
Please, keep support! |
|||
msg379942 - (view) | Author: Jorge Schrauwen (jorge.schrauwen) | Date: 2020-10-30 10:21 | |
I'm an illumos user too, mostly SmartOS and some OmniOS. I'm a heavy user of salt and thus phyton too, slowly prepping to get my stuff moved from 2.7 to 3.8 atm. As mentioned above parts of the illumos build also use python and are moving to wads 3.9. As mentioned I rarely use pip at all to install packages, nearly everything I need is available via IPS or pkgsrc depending on the distro. |
|||
msg379946 - (view) | Author: Olaf Bohlen (olafbohlen) | Date: 2020-10-30 10:51 | |
Also, let me join in here. OpenIndiana hipster and all other illumos distributions will suffer if you drop the "Solaris" support. Just to get an idea which packages in a nearly default install depend on python: pkg uninstall: Unable to remove 'runtime/python-35@3.5.9-2020.0.1.1' due to the following packages that depend on it: communication/im/pidgin@2.13.0-2020.0.1.4 desktop/compiz/ccsm@0.8.16-2020.0.1.0 desktop/time-slider@0.2.101-2020.0.1.116 desktop/xdg/menu-editor/mozo@1.24.0-2020.0.1.0 developer/build/onbld@0.5.11-2020.0.1.20038 developer/clang-80@8.0.1-2020.0.1.0 developer/ui-designer/glade@3.22.2-2020.0.1.0 developer/versioning/mercurial-35@5.3.2-2020.0.1.0 diagnostic/ddu@1.3.1-2020.0.1.40 gnome/config/gconf@2.32.4-2020.0.1.8 install/distribution-constructor@0.5.11-2020.0.1.1052 metapackages/build-essential@1.0-2020.0.1.21 package/pkg@0.5.11-2020.0.1.5543 package/pkg/system-repository@0.5.11-2020.0.1.5543 print/cups/system-config-printer@2.30.0-2020.0.1.3 print/filter/hplip@3.19.12-2020.0.1.0 release/os-welcome@1.0-2020.0.1.23 service/gnome/desktop-cache@0.2.2-2020.0.1.7 service/network/samba@4.12.5-2020.0.1.0 system/file-system/zfs@0.5.11-2020.0.1.20038 system/input-method/imf-startup@0.5.11-2020.0.1.3 system/install@0.5.11-2020.0.1.1052 system/network/avahi@0.7-2020.0.1.3 system/network/avahi/gui@0.7-2020.0.1.3 system/zones/brand/ipkg@0.5.11-2020.0.1.5543 text/texinfo@6.7-2020.0.1.1 web/server/apache-24/module/apache-wsgi-35@4.5.22-2020.0.1.5 Actually the package manager itself, installer, ZFS helpers, cups, the illumos build-system and and and all depend on python. |
|||
msg379955 - (view) | Author: Juraj Lutter (otis) | Date: 2020-10-30 12:40 | |
We, in danubecloud (https://github.com/erigones/esdc-ce) are relying on Python, too. Keep up the good work, folks :-) |
|||
msg379956 - (view) | Author: Mike La Spina (michael.laspina) | Date: 2020-10-30 12:47 | |
I suspect the download stats are not a full indication of usage on “SunOS” based on download stats. The bits are preinstalled. I do use it frequently via Illumos. Please support the platform. Regards, Mike |
|||
msg379959 - (view) | Author: Jan Poctavek (yanchii) | Date: 2020-10-30 12:57 | |
I'm speaking officially for Danube Cloud, an advanced project which is open-source virtualization platform similar to Proxmox, XCP-NG, oVirt, Joyent Triton, etc. Our base platform is SmartOS and we have everything written in Python. If you drop support for "Solaris", it will hurt our project a lot. Python has become heart of many projects and it should not limit itself to support just the mainstream platforms and OSes. Thank you for everything you've done so far and I hope we can continue together. Jan Poctavek Danube Cloud https://github.com/erigones/esdc-ce |
|||
msg379961 - (view) | Author: Gaige Paulsen (gaige) | Date: 2020-10-30 13:04 | |
Let me add my voice to those who ask to keep Solaris support in Python going forward. We use SmartOS which is an Illumos and thus Solaris derivative. We use Ansible extensively to automate our SmartOS deployments. We have a couple hundred VMs and 2 part-time volunteers on staff. We are extremely reliant on Ansible, and thus Python. I have multiple blogs and a couple of django-based web sites that are running on SmartOS and we run a wide variety of tools that use Python as part of their mix. |
|||
msg379962 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-30 13:06 | |
It seems like two Python core developers (Ronald Oussoren and Jesús Cea Avión) are volunteer to maintain the Solaris support. Moreover, it seems like some people would like to provide a Solaris server to run a buildbot server. In this case, I would be fine to keep the Solaris code. But it's not enough, please call for help: https://mail.python.org/archives/list/python-dev@python.org/message/NYWF4DUPFYZSKUUZ3ZHOSBQCWAF4IIQO/ |
|||
msg379963 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-30 13:06 | |
> But it's not enough, please call for help: Please *read* my call for help |
|||
msg379991 - (view) | Author: cyrus torros (sugondesenuts007) | Date: 2020-10-30 18:36 | |
Why?? I use python on solaris in my every day toolchain and at work. There is no harm in keeping it supported, it's not hard. Please do not drop support. |
|||
msg379992 - (view) | Author: Jakub Kulik (kulikjak) * | Date: 2020-10-30 18:51 | |
Hi, here I am passing on several notes from the Oracle Solaris team: - Oracle continues to develop and advance Oracle Solaris technology, and have committed to doing so for years to come. We release a new update (so called SRU) every month (https://blogs.oracle.com/solaris/oracle-solaris%3a-update-to-the-continuous-delivery-model). - Python is essential for Oracle Solaris and actively worked on. In fact, we recently finished migration to Python 3.7 and integrated Python 3.9, which will be released to customers soon. - Internally, we are performing daily testing for each Python version from 3.7+ and reporting any issues that are found* (fortunately, there are not many of them because Python works very well!). - Python 3 is running for more than a year now on any machine running Oracle Solaris 11.4. - Looking at six download statistics is very misleading because we are packaging Python and its modules in Solaris packages. This package is essential (meaning that every Oracle Solaris machine will have it installed). Also, we are caching archives internally, so unless customers download a newer version into a virtual environment, you won't see more than a few downloads from Oracle Solaris (which are then distributed onto thousands of machines running Oracle Solaris). - We are looking at how to provide build bots running on Oracle Solaris. - We are trying to be part of the Python community and also put back Solaris specific fixes (we provide patches for many of those issues reported). - It is certainly much better (hopefully for everyone) to resolve any specific Solaris issues and help with the maintenance within the Python community than do this just internally as part of Oracle Solaris development. *) everything reported by me are Solaris related issues Jakub |
|||
msg379993 - (view) | Author: Raymond Hettinger (rhettinger) * ![]() |
Date: 2020-10-30 18:57 | |
I think the PR needs to be withdrawn. Users have spoken. There is zero need to cut them off. |
|||
msg379994 - (view) | Author: Pablo Galindo Salgado (pablogsal) * ![]() |
Date: 2020-10-30 19:08 | |
Discussions are happening in this thread, including the requirements for supported platforms going forward: https://mail.python.org/archives/list/python-dev@python.org/thread/VDD7NMEDFXMOP4S74GEYJUHJRJPK2UR3/ |
|||
msg380006 - (view) | Author: Terry J. Reedy (terry.reedy) * ![]() |
Date: 2020-10-30 22:24 | |
Dear Solaris Python fans (about 20 so far here): Here is the situation. There are 1000s of open issues on this tracker and at least 100s of open cpython PRs, and only 20-30 core developers, mostly volunteers, actively (depending on definition) merging PRs and maybe another 10 triagers helping to manage issues. You all can help on issues by checking whether reported bugs still exist with current Python (3.8+) on current 'Solaris' (which includes what?). Searching open issues for 'Solaris' in 'All text' just returned 114 hits. About half were last touched over two years ago, and sometimes the last touch was inconsequential (a version or nosy change). I suspect many of the 114 are obsolete. For example, the last comment, five years ago, on #1471934, says the problem was fixed in Solaris 11.2. Does this mean the issue should be closed? The bottleneck for merging PRs is reviewing PRs. We coredevs cannot do enough reviews ourselves. But any competent user can help. Reviewing has two components. First, does the patch fix the problem? Testing this requires a Github account and a local clone and ability to build a test binary. See devguide.python.org for more. Second, does the patch meet our standards of code quality. Solaris-specific patches likely change the C part of cpython, so C competence and understanding of PEP 7 is needed here. |
|||
msg380008 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-10-30 22:30 | |
Jakub: Would you mind to build the master branch of Python, run the test suite and post the tests results? In short: git clone https://github.com/python/cpython.git cd cpython ./configure ./python -m test -j0 -r If possible, try to install dependencies (readline, lzma, etc.). If there are failures, please copy them into a file, and attach the file to this issue. It is to have an idea if it would be worth it to add a Python buildbot running on Solaris. If there are too many failures, it would be better to fix first most failures before considering to add a buildbot. |
|||
msg380080 - (view) | Author: Apostolos Syropoulos (ijdt.editor) | Date: 2020-10-31 17:02 | |
I think this is really a bad idea. I am using OpenIndiana in my classrooms and I teach Python! So if your drop Solaris support, it means I have to install a new supported OS on my many systems, to learn and teach many things, etc, This will make my life extremely difficult. |
|||
msg380196 - (view) | Author: Jakub Kulik (kulikjak) * | Date: 2020-11-02 08:18 | |
I ran the test and the results are attached (the first one is a complete test and the other one failed tests in verbose mode). I checked the failing tests and divided them into several groups: issues already reported: test_asyncio - reported and being solved here: https://bugs.python.org/issue38323 test_posix - problem with return values, reported here: https://bugs.python.org/issue41839 test_shutil - sendfile for Solaris was disabled in the library, but not in the test suite, which leads to the issues (I asked about reenabling here: https://bugs.python.org/issue41843; that would solve the issue) these failures are probably related to our internal network settings (false positives): test_ssl, test_urllib, test_urllib2, test_urllib2_localnet other: test_float - locale related * test_locale - locale related * test_re - most likely locale related * test_socket - known issues not yet reported upstream (not ready for acceptable PR) test_time - one locale related * and other not yet known test_tcl - yet to investigate *) Locale related failures are due to wchar_t differences between the Linux world and ours. It's something we resolved very recently and have yet to rework it into an upstreamable form and report it. It would be interesting to see Illumos results as well, because while also Solaris, there might be differences due to almost ten years of spit development. |
|||
msg380203 - (view) | Author: John Gardner (jgardner100) | Date: 2020-11-02 10:02 | |
Don't remove, it is still actively used by so many people. If the maintance overhead of 700 lines is a large burdern, then I'm happy to take on whatever work is required for it. |
|||
msg380205 - (view) | Author: Andy Fiddaman (omnios) | Date: 2020-11-02 10:55 | |
Jakub's results looks very familiar to me, having been working on python 3.9 on illumos. For OmniOS, we currently skip these tests via --ignorefile: # wchar_t related failures *.test_re.ReTests.test_locale_compiled *.test_re.ReTests.test_locale_caching # illumos does not support multiple SCM_RIGHTS messages in a packet *FDPassSeparate* # These tests fail on illumos and the first consumes a significant # amount of memory. Investigation required. test.test_socket.SendfileUsingSendfileTest.testCount test.test_socket.SendfileUsingSendfileTest.testWithTimeout test.test_socket.SendfileUsingSendfileTest.testOffset # test.test_asyncio.test_sendfile.* test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests.* We are also carrying some local patches for the following. Several come from Solaris. - Stop python detecting and using epoll() on illumos - scheduling priorities can be < 0 https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/mod-posix-sched_priority.patch - differences in sendfile behaviour (improves the situation but is not complete) https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/mod-posix-sendfile.patch - Enable sendfile for shutil.copy (mismatch between library and testsuite in terms of whether sendfile() is enabled on illumos) https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/mod-shutil-sendfile.patch - Fixes for building the socket module (_XOPEN_SOURCE=600) https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/mod-socket-xpg6.patch - Emulate clock_gettime(CLOCK_THREAD_CPUTIME_ID) https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/mod-time-threadtime.patch - PTY patch https://github.com/citrus-it/omnios-build/blob/python39/build/python39/patches/pty.patch and a few others, not all of which are suitable for upstream. https://github.com/citrus-it/omnios-build/tree/python39/build/python39/patches With all of this in place, the headline test stats for OmniOS Python 3.9 are: 401 tests OK. 24 tests skipped: test_dbm_gnu test_epoll test_gdb test_idle test_kqueue test_msilib test_ossaudiodev test_smtpnet test_socketserver test_startfile test_tcl test_timeout test_tix test_tk test_ttk_guionly test_ttk_textonly test_turtle test_urllib2net test_urllibnet test_winconsoleio test_winreg test_winsound test_xmlrpc_net test_zipfile64 Tests result: SUCCESS and, additionally, the dtrace tests succeed (we test them separately as they require elevated privileges). |
|||
msg380210 - (view) | Author: Jakub Kulik (kulikjak) * | Date: 2020-11-02 11:47 | |
Thanks Andy, good to know we are seeing the same issue! We tried to fix sendfile differences in Python code before as well, but as you said, it was never 100% solved. Recently we finally fixed it in C and it was accepted (https://bugs.python.org/issue41687) and backported to 3.9 so you might/should be ok without the patch as well. CLOCK_THREAD_CPUTIME_ID was issue for us as well but before the patch was accepted upstream, Oracle Solaris implemented it so it was no longer necessary. But the issue is still open https://bugs.python.org/issue35455 and knowing that others will use it, I will dust it off and finish it. Knowing that Oracle and Illumos are facing the same issues, I am much more confident that our patches won't break Illumos. I will start upstream more of them right away. |
|||
msg380241 - (view) | Author: Gordon Ross (gwr) | Date: 2020-11-02 17:45 | |
I can understand the frustrations around dealing with minority platforms, but please remember that the illumos project (www.illumos.org) is basically the inheritor of problems around "Build on Solaris" for 3rd party software like Python. There are several OS distributions based on illumos that would be impacted by removing the ability to (easily) build Python. I'm confident we can find some people to help maintain build-bots etc. if that's what it takes. If you're still looking for such help, let me know and I'll ask on developers@list.illumos.org for volunteers. Thanks! -GWR |
|||
msg380256 - (view) | Author: Terry J. Reedy (terry.reedy) * ![]() |
Date: 2020-11-02 23:29 | |
People interested in helping Solaris issues should see my post above, msg380006. Issue #35455, about thread time on Solaris, with a simple but non-trivial patch, was just closed as fixed after reviews from 2 coredevs and a 3rd person. I presume Victor will close his proposed PR when there is a buildbot and additional action on issues. We have gotten new information that supports doing so. |
|||
msg380257 - (view) | Author: Raymond Hettinger (rhettinger) * ![]() |
Date: 2020-11-02 23:44 | |
Victor, are you going to close the PR? |
|||
msg380455 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-11-06 16:32 | |
Copy of my email sent to python-dev. https://mail.python.org/archives/list/python-dev@python.org/message/46UUJ4J5YLCWF2JQMC5L7OIYG6PNQLRL/ Since I created the issue and the PR, and sent this email to python-dev (one week ago), many Solaris and Solaris-like (ex: Illumos) users replied that the operating system is definitively alive. I didn't know that Oracle still ships new Solaris updates every month: that's a good thing! But this is not enough to support a platform. We would need proactive contributors to fix known Solaris issues, but also fix new Solaris issues (either regressions, or bugs newly discovered). We would also need a buildbot to run the Python test suite on Solaris (or again, a Solaris-like OS). The good news is that Jakub Kulik started to fix some Solaris issues. I understood that Solaris and Solaris-like operating systems do have downstream patches on Python to fix a bunch of bugs. It seems like some people want to push these fixes to Python upstream which is also a good sign. The other problem that I wanted to discuss is that fixing Solaris issues require core devs (who merge PRs) accessing Solaris. If contributors send patches and some core devs are fine with merging fixes without being able to test them manually, I'm also fine with that. My first intent was to remove support for a definitely dead operating system, but it seems like I was completely wrong (it's alive!). Thanks to people starting to fix Solaris issues, I close my PR and I no longer plan to drop Solaris support. I prefer to leave bpo-42173 open for now, since people decide to use it as a place to collaborate on fixing Solaris issues. Once most tests will pass on the master branch, I also hope that someone will set up a buildbot *and* fix issues discovered by this buildbot. Sorry but just setting up a buildbot doesn't solve any problem, it only increases the maintenance burden for people who maintain the buildbot fleet. For example, we have two AIX buildbots, I report bugs on bugs.python.org, but it seems like nobody is available to fix them... Overall, I'm quite happy with what is happening with Solaris! More collaboration, issues being fixed in Python upstream. I just hope that this work will continue next months. ;-) |
|||
msg380456 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-11-06 16:33 | |
This issue title is "Drop Solaris support". I no longer plan to drop Solaris support. I suggest to open new issues and continue the discussion there. |
|||
msg380457 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-11-06 16:37 | |
Gaige B Paulsen offered to set up a SmartOS (Illumos-derived) buildbot: https://mail.python.org/archives/list/python-buildbots@python.org/thread/7HGPHD3WZG6V6NVP6EGQO6NHZ3DAPPXN/ |
|||
msg381122 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2020-11-16 17:05 | |
I close the issue. Thanks for the constructive discussion and I hope that more and more downstream patches will be merged in Python upstream ;-) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:37 | admin | set | github: 86339 |
2020-11-16 17:05:24 | vstinner | set | status: open -> closed resolution: not a bug messages: + msg381122 stage: patch review -> resolved |
2020-11-06 16:37:59 | vstinner | set | messages: + msg380457 |
2020-11-06 16:33:22 | vstinner | set | messages: + msg380456 |
2020-11-06 16:32:11 | vstinner | set | messages: + msg380455 |
2020-11-05 16:21:06 | jstasiak | set | nosy:
+ jstasiak |
2020-11-02 23:44:51 | rhettinger | set | messages: + msg380257 |
2020-11-02 23:29:10 | terry.reedy | set | messages: + msg380256 |
2020-11-02 17:45:19 | gwr | set | nosy:
+ gwr messages: + msg380241 |
2020-11-02 11:47:08 | kulikjak | set | messages: + msg380210 |
2020-11-02 10:55:38 | omnios | set | messages: + msg380205 |
2020-11-02 10:02:44 | jgardner100 | set | nosy:
+ jgardner100 messages: + msg380203 |
2020-11-02 08:18:43 | kulikjak | set | files: + Oracle_Solaris_detailed_test.txt |
2020-11-02 08:18:33 | kulikjak | set | files:
+ Oracle_Solaris_full_test.txt messages: + msg380196 |
2020-11-01 14:56:12 | jomasoftmarcel | set | nosy:
+ jomasoftmarcel |
2020-10-31 17:02:52 | ijdt.editor | set | nosy:
+ ijdt.editor messages: + msg380080 |
2020-10-30 22:30:47 | vstinner | set | messages: + msg380008 |
2020-10-30 22:24:14 | terry.reedy | set | nosy:
+ terry.reedy messages: + msg380006 |
2020-10-30 19:08:52 | pablogsal | set | messages: + msg379994 |
2020-10-30 18:57:38 | rhettinger | set | nosy:
+ rhettinger messages: + msg379993 |
2020-10-30 18:51:23 | kulikjak | set | nosy:
+ kulikjak, - sugondesenuts007 messages: + msg379992 |
2020-10-30 18:36:59 | sugondesenuts007 | set | nosy:
+ sugondesenuts007 messages: + msg379991 |
2020-10-30 13:06:52 | vstinner | set | messages: + msg379963 |
2020-10-30 13:06:27 | vstinner | set | messages: + msg379962 |
2020-10-30 13:04:53 | gaige | set | nosy:
+ gaige messages: + msg379961 |
2020-10-30 12:57:35 | yanchii | set | nosy:
+ yanchii messages: + msg379959 |
2020-10-30 12:47:06 | michael.laspina | set | nosy:
+ michael.laspina messages: + msg379956 |
2020-10-30 12:40:33 | otis | set | nosy:
+ otis messages: + msg379955 |
2020-10-30 10:51:36 | olafbohlen | set | nosy:
+ olafbohlen messages: + msg379946 |
2020-10-30 10:21:51 | jorge.schrauwen | set | nosy:
+ jorge.schrauwen messages: + msg379942 |
2020-10-30 08:46:16 | predrag.zecevic.1961 | set | nosy:
+ predrag.zecevic.1961 messages: + msg379938 versions: - Python 3.10 |
2020-10-30 08:44:02 | abarbu | set | nosy:
+ abarbu messages: + msg379937 |
2020-10-30 08:17:59 | marcheschi | set | nosy:
+ marcheschi messages: + msg379933 |
2020-10-30 07:37:20 | cgrzemba | set | nosy:
+ cgrzemba messages: + msg379927 |
2020-10-30 07:33:41 | siepkes | set | nosy:
+ siepkes messages: + msg379926 |
2020-10-30 07:28:01 | mariuspana | set | nosy:
+ mariuspana messages: + msg379925 |
2020-10-30 07:21:04 | tbalbers | set | nosy:
+ tbalbers messages: + msg379924 |
2020-10-30 04:24:13 | jm650 | set | nosy:
+ jm650 messages: + msg379915 |
2020-10-30 03:28:51 | robertdfrench | set | nosy:
+ robertdfrench messages: + msg379910 |
2020-10-30 01:44:07 | brett3 | set | nosy:
+ brett3 messages: + msg379906 |
2020-10-30 01:17:54 | alanc | set | nosy:
+ alanc |
2020-10-30 00:47:00 | pfirmsto | set | nosy:
+ pfirmsto messages: + msg379901 |
2020-10-29 23:39:32 | tropikhajma@gmail.com | set | nosy:
+ tropikhajma@gmail.com messages: + msg379900 |
2020-10-29 23:06:31 | omnios | set | messages: + msg379899 |
2020-10-29 22:59:14 | omnios | set | nosy:
+ omnios messages: + msg379897 |
2020-10-29 22:48:27 | jclulow | set | nosy:
+ jclulow messages: + msg379895 |
2020-10-29 22:45:29 | jcea | set | messages: + msg379894 |
2020-10-29 22:38:57 | jcea | set | messages: + msg379893 |
2020-10-29 22:36:38 | mimi.vx | set | nosy:
+ mimi.vx messages: + msg379891 |
2020-10-29 22:22:17 | wiedi | set | messages: + msg379890 |
2020-10-29 22:14:26 | vstinner | set | messages: + msg379888 |
2020-10-29 22:13:29 | vstinner | set | messages: + msg379887 |
2020-10-29 22:13:07 | wiedi | set | nosy:
+ wiedi messages: + msg379886 |
2020-10-28 23:46:51 | Yonatan Goldschmidt | set | nosy:
+ Yonatan Goldschmidt |
2020-10-27 16:02:24 | vstinner | set | keywords:
+ patch stage: patch review pull_requests: + pull_request21917 |
2020-10-27 16:01:47 | BTaskaya | set | nosy:
+ BTaskaya |
2020-10-27 15:59:08 | vstinner | create |