Author ncoghlan
Recipients BreamoreBoy, NaCl, bwanamarko, larry, ncoghlan, ned.deily, paul.moore, r.david.murray, steve.dower, terry.reedy, tim.golden, zach.ware
Date 2015-03-22.00:51:31
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1426985492.4.0.0447827827859.issue22516@psf.upfronthosting.co.za>
In-reply-to
Content
As of last month, Microsoft do provide official Python support, but only in the context of the online Azure Machine Learning Environment: http://blogs.technet.com/b/machinelearning/archive/2015/02/18/announcing-the-general-availability-of-azure-machine-learning.aspx

I only discovered that this morning myself when Doug Napoleone tweeted a link to the related post from Continuum Analytics: http://continuum.io/blog/azureml

I was already aware of another Strata "combined solution" announcement, which covered deploying Anaconda Cluster in combination with Red Hat Storage for distributed data access from PySpark analysis nodes: http://redhatstorage.redhat.com/2015/02/17/deploying-pyspark-on-red-hat-storage-glusterfs/

So if we're after a recommendation for end users that has strong multi-vendor backing, then Anaconda from Continuum Analytics would definitely be the way to go. It's also already the preferred choice of the Software Carpentry community: http://software-carpentry.org/v5/setup.html (and the reasons are the same as those being discussed here: unlike the default CPython installers, Anaconda is specifically designed to tolerate being installed without administrative access to your system)

(I'd previously discounted Anaconda as a default upstream recommendation for per-user installs on Windows as I thought it was missing Windows-specific utility libraries like pywin32, but I was simply wrong about that: http://continuum.io/press/anaconda-1-6-released)

As Steve noted, providing Python *in the OS* for third party use poses major upgrade challenges. For Fedora, Python gets rebased in each major release (so every 6 months or so) which is easily fast enough to keep up with the general pace of CPython releases (Ubuntu tend to do the same thing), but also means system administrators and other folks using the system Python need to be prepared to keep pace with that upgrade cycle.

Things get far more difficult with the long term support releases like Debian Stable, Ubuntu LTS, RHEL, CentOS and SLES, or the custom Linux distributions used inside organisations like Google, Facebook and Amazon. *There* the system Python has historically acted as a long term brake on Python's evolution, as third party projects are often reluctant to drop support for older Python versions until newer Python versions are readily available in these long term support releases.

In July, we'll be coming up on 5 years since Python 2.7 was released, yet a great many third party Python projects still require that contributions retain compatibility with Python 2.6.

We see similar patterns occurring if we look back at earlier Python 2.x releases. As part of explaining the limited utility of a Python 2.8 release in facilitating the Python 3 transition (see
http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html#wouldn-t-a-python-2-8-release-help-ease-the-transition for more on that) I pulled together the dates from various Python 2.x releases to when Twisted dropped the preceding release as a compatibility requirement:

* CPython 2.4 released -> 2.3 support dropped in Twisted: ~6 years
* CPython 2.5 released -> 2.4 support dropped in Twisted: ~4 years (Windows), ~6 years (non-Windows)
* CPython 2.6 released -> 2.5 support dropped in Twisted: ~4 years

That's a broken cycle, and one we need to help fix on the distro side by getting people to migrate away from using the stable system Python (which won't be rebased for the entire lifecycle of a major release) and into more regularly updated environments - upstream can't do that for us, we have to offer the appropriate tools downstream and actively encourage users to leave the system Python to the task of running the system, rather than arbitrary third party code.

The 10 year supported lifespan of RHEL & CentOS, together with the 3-4 year gaps between new major releases is one of the worst offenders on that front (not just for Python, but for other language runtimes, database runtimes, web servers etc), hence initiatives like the introduction of Software Collections (both as a commercially supported Red Hat product and as an upstream project hosted at softwarecollections.org). Using an SCL instead of the system Python lets an application developer more easily upgrade at their own pace (whether faster on RHEL/CentOS or slower on Fedora)
History
Date User Action Args
2015-03-22 00:51:32ncoghlansetrecipients: + ncoghlan, terry.reedy, paul.moore, larry, tim.golden, ned.deily, r.david.murray, BreamoreBoy, zach.ware, steve.dower, bwanamarko, NaCl
2015-03-22 00:51:32ncoghlansetmessageid: <1426985492.4.0.0447827827859.issue22516@psf.upfronthosting.co.za>
2015-03-22 00:51:32ncoghlanlinkissue22516 messages
2015-03-22 00:51:31ncoghlancreate