PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments #46628

glyph · 2008-03-18T00:34:09Z

BPO	2375
Nosy	@malemburg, @birkenfeld, @abalkin, @benjaminp, @glyph, @bitdancer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-01-14.19:03:13.463>
created_at = <Date 2008-03-18.00:34:08.768>
labels = ['interpreter-core', 'type-feature']
title = 'PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments'
updated_at = <Date 2010-01-14.19:03:13.462>
user = 'https://github.com/glyph'

bugs.python.org fields:

activity = <Date 2010-01-14.19:03:13.462>
actor = 'lemburg'
assignee = 'nnorwitz'
closed = True
closed_date = <Date 2010-01-14.19:03:13.463>
closer = 'lemburg'
components = ['Interpreter Core']
creation = <Date 2008-03-18.00:34:08.768>
creator = 'glyph'
dependencies = []
files = []
hgrepos = []
issue_num = 2375
keywords = []
message_count = 17.0
messages = ['63820', '63841', '64046', '64057', '64095', '64448', '64449', '64589', '64591', '70469', '97724', '97726', '97727', '97728', '97729', '97730', '97782']
nosy_count = 8.0
nosy_names = ['lemburg', 'nnorwitz', 'georg.brandl', 'belopolsky', 'schmir', 'benjamin.peterson', 'glyph', 'r.david.murray']
pr_nums = []
priority = 'critical'
resolution = 'rejected'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue2375'
versions = ['Python 3.2']

glyph · 2008-03-18T00:34:08Z

Currently if you have both Python 3 and Python 2 installed, there's no
way to indicate that ".py" files in one area are Python 2 syntax and in
another area are Python 3 syntax. Being able to distinguish between
these would be nice for heterogeneous deployment environments, but is
particularly important for developers who are working on Python 3
conversions.

It would be good to have a PYTHON3PATH which, if present, would be
honored instead of PYTHONPATH. PYTHONPATH could still be honored if
PYTHON3PATH was not present, so this is purely a new feature and
shouldn't have much impact on compatibility.

abalkin · 2008-03-18T01:46:55Z

-1

Sites that need this functionality can emulate this feature in site.py
by adding

sys.path[1:1] = os.getenv("PYTHON3PATH", "").split(os.pathsep)

in py3k installation.

I could not find any discussion beyond the original post at

http://mail.python.org/pipermail/python-3000/2008-February/012008.html

glyph · 2008-03-19T07:38:53Z

I don't understand your objection. It sounds like you're objecting, but
then suggesting an implementation? The line you suggest might just as
easily be present in py3k's default site.py and it would work fine.

As I understand it, "sites" don't need this feature, developers do.
Let's say I'm working on a typical system with stock python2 and python3
packages installed. Maybe it's a multi-user system, I don't have
control over site.py.

Normally what I'd do to manipulate sys.path dynamically between
different versions would be to install a sitecustomize.py on my one
PYTHONPATH entry that detected the version and reacted appropriately.
However, writing a program that is both valid python 2 and python 3 is,
as I understand it, both tricky and unsupported.

Will this specific case be supported so that developers don't need to
have root and modify the system python in order to do 2->3 migrations?

abalkin · 2008-03-19T14:47:24Z

I don't understand your objection.
It sounds like you're objecting, but
then suggesting an implementation?

This sounds like a kludge addressing a transient
problem. It also feels like a solution catering
to developers at the expense of user confusion:

Why is there PYTHON3PATH, but not PYTHON2PATH?
Is PYTHONPATH effectively PYTHON2PATH or PYTHON1PATH?
Will PYTHON3PATH be supported forever or deprecated once 2.8 and 3.2
are close enough to support a large common code base?

There are many ways to address multi-python site issues. PYTHONPATH is
too indiscriminate for some uses. When more fine-grained
differentiation between 3.x and 2.x files is needed, a better approach
would be a custom loader in metapath that can look for version specific
file extensions.

I would suggest to wait until there are enough reports from the field on
different solutions in actual use before selecting one to standardize
on.

[I have to mention that perl 5 uses PERL5PATH, but I am not sure whether
it is an argument for or against PYTHON3PATH. Can anyone report on pros
and cons of PERL5PATH from actual experience?]

glyph · 2008-03-19T20:32:23Z

The idea is that PYTHON3PATH will be honored in preference to
PYTHONPATH, but PYTHONPATH will still be honored. It's exposed only to
people who specifically need it.

However, I think it's misleading to call the Python 3 transition a
"transient" problem. If we're lucky, it's going to be a decade of slow
progress. If we're not, it will never end. Try, for example, compiling
your system C library without "gets", and see what happens. (And that's
been an issue for more than 10 years!)

The biggest way to shorten this problem is to provide tools and idioms
for developers to use when porting. Specifically like this feature.

As far as "version specific file extensions" - I'd be very happy with
that but Guido has said several times that he doesn't like it. Feel
free to discuss that with him, but this change is far less invasive.
Users who don't need it will simply never know about it.

schmir · 2008-03-24T22:48:03Z

I once even had the need for a PYTHON24PATH or PYTHON25PATH.
(And I think this would be the right approach, i.e. even using the minor
version number).
However, now I am a happy user of virtualenv, and I do not even have to
set PYTHONPATH anymore...

schmir · 2008-03-24T22:49:43Z

c modules cannot be used when the minor version number changes. If I
remember correctly c modules where the problem for me, when I ran python
2.4 and 2.5 in parallel.

birkenfeld · 2008-03-27T13:17:23Z

I even run modules compiled for Python 2.2 successfully on 2.5...

abalkin · 2008-03-27T14:05:18Z

I have never had a problem of different python versions coexisting on
the same machine, but having 32-bit and 64-bit python coexist is much
harder. Particularly when 32-bit python is compiled on the 64-bit OS
(using -m32 flag). There is a related bpo-1294959 highlighting this
problem.

See also bpo-1536339, bpo-1553166, and bpo-858809.

Ideally, I would like to see a mechanism that would allow both standard
library and user modules to share machine independent (*.py{,c,o}) files
while having separate locations for 32-bit and 64-bit C modules.

benjaminp · 2008-07-31T02:04:34Z

What's the status of this?

malemburg · 2010-01-13T15:20:11Z

Setting up specific environments for each Python version is outside the scope of Python.

This is something the user needs to handle using a virtualenv setup, an env-setup shell script or similar approach.

bitdancer · 2010-01-13T17:15:49Z

I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'. This is unlike the superficially similar situation with different versions of the 2 line.

I think this issue should be discussed on python-dev before being closed, and will post a pointer there.

malemburg · 2010-01-13T17:30:53Z

R. David Murray wrote:

R. David Murray <rdmurray@bitdance.com> added the comment:

I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'. This is unlike the superficially similar situation with different versions of the 2 line.

I think this issue should be discussed on python-dev before being closed, and will post a pointer there.

You can easily have python3 implemented as shell script setting up
PYTHONPATH to whatever your particular Python installation uses.

Python itself does not install as python3, so this is not a configuration
supported by Python anyway.

Also note that it's easy to write a sitecustomize.py which takes
care of whatever platform specific side-by-side installation
you want to support with an OS or distribution.

bitdancer · 2010-01-13T17:46:39Z

Yes, it does:

rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328
-rwxr-xr-x 1 rdmurray rdmurray 131 Dec 20 12:22 2to3
-rwxr-xr-x 1 rdmurray rdmurray 119 Dec 20 12:22 idle3
-rwxr-xr-x 1 rdmurray rdmurray 104 Dec 20 12:22 pydoc3
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3
lrwxrwxrwx 1 rdmurray rdmurray 16 Jan 13 12:44 python3-config -> python3.2-config
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2
-rwxr-xr-x 1 rdmurray rdmurray 1421 Jan 13 12:44 python3.2-config

That's a straight 'make install' from my svn checkout of the py3k trunk.

schmir · 2010-01-13T17:48:17Z

Marc-Andre Lemburg <mal@egenix.com> added the comment:

You can easily have python3 implemented as shell script setting up
PYTHONPATH to whatever your particular Python installation uses.

yes, and you need to write a python shell script, which resets that
variable in case you're starting a python 2 script from your python3
shell script.

malemburg · 2010-01-13T18:18:48Z

R. David Murray wrote:

R. David Murray <rdmurray@bitdance.com> added the comment:

Yes, it does:

rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328
-rwxr-xr-x 1 rdmurray rdmurray 131 Dec 20 12:22 2to3
-rwxr-xr-x 1 rdmurray rdmurray 119 Dec 20 12:22 idle3
-rwxr-xr-x 1 rdmurray rdmurray 104 Dec 20 12:22 pydoc3
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3
lrwxrwxrwx 1 rdmurray rdmurray 16 Jan 13 12:44 python3-config -> python3.2-config
-rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2
-rwxr-xr-x 1 rdmurray rdmurray 1421 Jan 13 12:44 python3.2-config

That's a straight 'make install' from my svn checkout of the py3k trunk.

Sorry, forgot about that change. The name of the binaries is changed
during installation - after running 'make' you get a 'python' binary.

I wonder whether anyone has ever tried building Python on AIX.
Renaming binaries after linking is a bad idea on AIX, since the
linker then sometimes fails to resolve symbols.

I'm still -1 on the change.

People setting up PYTHONPATH will know what they are doing and also
know which Python version they are going to use with the customized
setting and after the transition from Python 2 to 3 we'd be stuck
with user environment setups using PYTHON3PATH instead of PYTHONPATH,
so removing PYTHON3PATH again won't be easily possible.

Besides, if you change PYTHONPATH, you'd also have to change
PYTHONHOME, PYTHONSTARTUP, and all the other flags to use
the 'PYTHON3' prefix.

malemburg · 2010-01-14T19:03:13Z

Guido pronounced on this on python-dev, so closing the request again.

glyph mannequin added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Mar 18, 2008

nnorwitz mannequin self-assigned this Mar 18, 2008

benjaminp added the type-feature A feature request or enhancement label Mar 24, 2008

malemburg closed this as completed Jan 13, 2010

bitdancer reopened this Jan 13, 2010

malemburg closed this as completed Jan 14, 2010

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments #46628

PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments #46628

glyph mannequin commented Mar 18, 2008

glyph mannequin commented Mar 18, 2008

abalkin commented Mar 18, 2008

glyph mannequin commented Mar 19, 2008

abalkin commented Mar 19, 2008

glyph mannequin commented Mar 19, 2008

schmir mannequin commented Mar 24, 2008

schmir mannequin commented Mar 24, 2008

birkenfeld commented Mar 27, 2008

abalkin commented Mar 27, 2008

benjaminp commented Jul 31, 2008

malemburg commented Jan 13, 2010

bitdancer commented Jan 13, 2010

malemburg commented Jan 13, 2010

bitdancer commented Jan 13, 2010

schmir mannequin commented Jan 13, 2010

malemburg commented Jan 13, 2010

malemburg commented Jan 14, 2010

PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments #46628

PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments #46628

Comments

glyph mannequin commented Mar 18, 2008

glyph mannequin commented Mar 18, 2008

abalkin commented Mar 18, 2008

glyph mannequin commented Mar 19, 2008

abalkin commented Mar 19, 2008

glyph mannequin commented Mar 19, 2008

schmir mannequin commented Mar 24, 2008

schmir mannequin commented Mar 24, 2008

birkenfeld commented Mar 27, 2008

abalkin commented Mar 27, 2008

benjaminp commented Jul 31, 2008

malemburg commented Jan 13, 2010

bitdancer commented Jan 13, 2010

malemburg commented Jan 13, 2010

bitdancer commented Jan 13, 2010

schmir mannequin commented Jan 13, 2010

malemburg commented Jan 13, 2010

malemburg commented Jan 14, 2010