classification
Title: PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments
Type: enhancement Stage: needs patch
Components: Interpreter Core Versions: Python 3.2
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: belopolsky, benjamin.peterson, georg.brandl, glyph, lemburg, nnorwitz, r.david.murray, schmir
Priority: critical Keywords:

Created on 2008-03-18 00:34 by glyph, last changed 2010-01-14 19:03 by lemburg. This issue is now closed.

Messages (17)
msg63820 - (view) Author: Glyph Lefkowitz (glyph) (Python triager) Date: 2008-03-18 00:34
Currently if you have both Python 3 and Python 2 installed, there's no
way to indicate that ".py" files in one area are Python 2 syntax and in
another area are Python 3 syntax.  Being able to distinguish between
these would be nice for heterogeneous deployment environments, but is
particularly important for developers who are working on Python 3
conversions.

It would be good to have a PYTHON3PATH which, if present, would be
honored instead of PYTHONPATH.  PYTHONPATH could still be honored if
PYTHON3PATH was not present, so this is purely a new feature and
shouldn't have much impact on compatibility.
msg63841 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-03-18 01:46
-1

Sites that need this functionality can emulate this feature in site.py 
by adding

sys.path[1:1] = os.getenv("PYTHON3PATH", "").split(os.pathsep)

in py3k installation.

I could not find any discussion beyond the original post at

http://mail.python.org/pipermail/python-3000/2008-February/012008.html
msg64046 - (view) Author: Glyph Lefkowitz (glyph) (Python triager) Date: 2008-03-19 07:38
I don't understand your objection.  It sounds like you're objecting, but
then suggesting an implementation?  The line you suggest might just as
easily be present in py3k's default site.py and it would work fine.

As I understand it, "sites" don't need this feature, developers do. 
Let's say I'm working on a typical system with stock python2 and python3
packages installed.  Maybe it's a multi-user system, I don't have
control over site.py.

Normally what I'd do to manipulate sys.path dynamically between
different versions would be to install a sitecustomize.py on my one
PYTHONPATH entry that detected the version and reacted appropriately. 
However, writing a program that is both valid python 2 and python 3 is,
as I understand it, both tricky and unsupported.

Will this specific case be supported so that developers don't need to
have root and modify the system python in order to do 2->3 migrations?
msg64057 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-03-19 14:47
> I don't understand your objection.
> It sounds like you're objecting, but
> then suggesting an implementation?

This sounds like a kludge addressing a transient
problem.  It also feels like a solution catering
to developers at the expense of user confusion:

1. Why is there PYTHON3PATH, but not PYTHON2PATH?
2. Is PYTHONPATH effectively PYTHON2PATH or PYTHON1PATH?
3. Will PYTHON3PATH be supported forever or deprecated once 2.8 and 3.2 
are close enough to support a large common code base?

There are many ways to address multi-python site issues. PYTHONPATH is 
too indiscriminate for some uses.  When more fine-grained 
differentiation between 3.x and 2.x files is needed, a better approach 
would be a custom loader in metapath that can look for version specific 
file extensions.

I would suggest to wait until there are enough reports from the field on 
different solutions in actual use before selecting one to standardize 
on.

[I have to mention that perl 5 uses PERL5PATH, but I am not sure whether 
it is an argument for or against PYTHON3PATH.  Can anyone report on pros 
and cons of PERL5PATH from actual experience?]
msg64095 - (view) Author: Glyph Lefkowitz (glyph) (Python triager) Date: 2008-03-19 20:32
The idea is that PYTHON3PATH will be honored in preference to
PYTHONPATH, but PYTHONPATH will still be honored.  It's exposed only to
people who specifically need it.

However, I think it's misleading to call the Python 3 transition a
"transient" problem.  If we're lucky, it's going to be a decade of slow
progress.  If we're not, it will never end.  Try, for example, compiling
your system C library without "gets", and see what happens.  (And that's
been an issue for more than 10 years!)

The biggest way to shorten this problem is to provide tools and idioms
for developers to use when porting.  Specifically like this feature.

As far as "version specific file extensions" - I'd be very happy with
that but Guido has said several times that he doesn't like it.  Feel
free to discuss that with him, but this change is far less invasive. 
Users who don't need it will simply never know about it.
msg64448 - (view) Author: Ralf Schmitt (schmir) Date: 2008-03-24 22:48
I once even had the need for a PYTHON24PATH or PYTHON25PATH.
(And I think this would be the right approach, i.e. even using the minor
version number).
However, now I am a happy user of virtualenv, and I do not even have to
set PYTHONPATH anymore...
msg64449 - (view) Author: Ralf Schmitt (schmir) Date: 2008-03-24 22:49
c modules cannot be used when the minor version number changes. If I
remember correctly c modules where the problem for me, when I ran python
2.4 and 2.5 in parallel.
msg64589 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-03-27 13:17
I even run modules compiled for Python 2.2 successfully on 2.5...
msg64591 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-03-27 14:05
I have never had a problem of different python versions coexisting on 
the same machine, but having 32-bit and 64-bit python coexist is much 
harder. Particularly when 32-bit python is compiled on the 64-bit OS 
(using -m32 flag).  There is a related issue1294959 highlighting this 
problem.

See also issue1536339, issue1553166, and issue858809.

Ideally, I would like to see a mechanism that would allow both standard 
library and user modules to share machine independent (*.py{,c,o}) files 
while having separate locations for 32-bit and 64-bit C modules.
msg70469 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-07-31 02:04
What's the status of this?
msg97724 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2010-01-13 15:20
Setting up specific environments for each Python version is outside the scope of Python.

This is something the user needs to handle using a virtualenv setup, an env-setup shell script or similar approach.
msg97726 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-01-13 17:15
I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'.  This is unlike the superficially similar situation with different versions of the 2 line.

I think this issue should be discussed on python-dev before being closed, and will post a pointer there.
msg97727 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2010-01-13 17:30
R. David Murray wrote:
> 
> R. David Murray <rdmurray@bitdance.com> added the comment:
> 
> I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'.  This is unlike the superficially similar situation with different versions of the 2 line.
> 
> I think this issue should be discussed on python-dev before being closed, and will post a pointer there.

You can easily have python3 implemented as shell script setting up
PYTHONPATH to whatever your particular Python installation uses.

Python itself does not install as python3, so this is not a configuration
supported by Python anyway.

Also note that it's easy to write a sitecustomize.py which takes
care of whatever platform specific side-by-side installation
you want to support with an OS or distribution.
msg97728 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-01-13 17:46
Yes, it does:

rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328
-rwxr-xr-x 1 rdmurray rdmurray     131 Dec 20 12:22 2to3
-rwxr-xr-x 1 rdmurray rdmurray     119 Dec 20 12:22 idle3
-rwxr-xr-x 1 rdmurray rdmurray     104 Dec 20 12:22 pydoc3
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3
lrwxrwxrwx 1 rdmurray rdmurray      16 Jan 13 12:44 python3-config -> python3.2-config
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2
-rwxr-xr-x 1 rdmurray rdmurray    1421 Jan 13 12:44 python3.2-config

That's a straight 'make install' from my svn checkout of the py3k trunk.
msg97729 - (view) Author: Ralf Schmitt (schmir) Date: 2010-01-13 17:48
> Marc-Andre Lemburg <mal@egenix.com> added the comment:
>
>
> You can easily have python3 implemented as shell script setting up
> PYTHONPATH to whatever your particular Python installation uses.

yes, and you need to write a python shell script, which resets that
variable in case you're starting a python 2 script from your python3
shell script.
msg97730 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2010-01-13 18:18
R. David Murray wrote:
> 
> R. David Murray <rdmurray@bitdance.com> added the comment:
> 
> Yes, it does:
> 
> rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328
> -rwxr-xr-x 1 rdmurray rdmurray     131 Dec 20 12:22 2to3
> -rwxr-xr-x 1 rdmurray rdmurray     119 Dec 20 12:22 idle3
> -rwxr-xr-x 1 rdmurray rdmurray     104 Dec 20 12:22 pydoc3
> -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3
> lrwxrwxrwx 1 rdmurray rdmurray      16 Jan 13 12:44 python3-config -> python3.2-config
> -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2
> -rwxr-xr-x 1 rdmurray rdmurray    1421 Jan 13 12:44 python3.2-config
> 
> That's a straight 'make install' from my svn checkout of the py3k trunk.

Sorry, forgot about that change. The name of the binaries is changed
during installation - after running 'make' you get a 'python' binary.

I wonder whether anyone has ever tried building Python on AIX.
Renaming binaries after linking is a bad idea on AIX, since the
linker then sometimes fails to resolve symbols.

I'm still -1 on the change.

People setting up PYTHONPATH will know what they are doing and also
know which Python version they are going to use with the customized
setting and after the transition from Python 2 to 3 we'd be stuck
with user environment setups using PYTHON3PATH instead of PYTHONPATH,
so removing PYTHON3PATH again won't be easily possible.

Besides, if you change PYTHONPATH, you'd also have to change
PYTHONHOME, PYTHONSTARTUP, and all the other flags to use
the 'PYTHON3' prefix.
msg97782 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2010-01-14 19:03
Guido pronounced on this on python-dev, so closing the request again.
History
Date User Action Args
2010-01-14 19:03:13lemburgsetstatus: open -> closed

messages: + msg97782
2010-01-13 18:18:49lemburgsetmessages: + msg97730
2010-01-13 17:48:17schmirsetmessages: + msg97729
2010-01-13 17:46:39r.david.murraysetmessages: + msg97728
2010-01-13 17:30:52lemburgsetmessages: + msg97727
2010-01-13 17:15:49r.david.murraysetstatus: closed -> open
nosy: + r.david.murray
messages: + msg97726

2010-01-13 15:20:10lemburgsetstatus: open -> closed

nosy: + lemburg
messages: + msg97724

resolution: rejected
2010-01-13 01:37:09brian.curtinsetstage: needs patch
versions: + Python 3.2, - Python 3.0
2010-01-13 01:34:32brian.curtinlinkissue3636 superseder
2008-07-31 02:04:33benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg70469
2008-03-27 14:05:18belopolskysetmessages: + msg64591
2008-03-27 13:17:23georg.brandlsetnosy: + georg.brandl
messages: + msg64589
2008-03-24 22:51:49benjamin.petersonsettype: enhancement
2008-03-24 22:49:43schmirsetmessages: + msg64449
2008-03-24 22:48:03schmirsetnosy: + schmir
messages: + msg64448
2008-03-19 20:32:24glyphsetmessages: + msg64095
2008-03-19 14:47:24belopolskysetmessages: + msg64057
2008-03-19 07:38:53glyphsetmessages: + msg64046
2008-03-18 01:46:55belopolskysetnosy: + belopolsky
messages: + msg63841
2008-03-18 00:37:07nnorwitzsetpriority: critical
assignee: nnorwitz
nosy: + nnorwitz
2008-03-18 00:34:08glyphcreate