msg389489 - (view) |
Author: Jared Sutton (jpsutton) * |
Date: 2021-03-24 20:35 |
The behavior of os.path.join() does not match the documentation, in regards to the use of os.sep. From the docs:
"""
The return value is the concatenation of path and any members of *paths with exactly one directory separator (os.sep) following each non-empty part except the last, meaning that the result will only end in a separator if the last part is empty.
"""
The documentation clearly states that the function uses the value of os.sep (which differs based on platform). However, if you review the 2 implementations (ntpath.py and posixpath.py), the separator character used is clearly hard-coded and doesn't reference os.sep at all.
One could say that this is either a doc bug or an implementation bug, depending on what the intended behavior is. I submit that this is an implementation bug, as one might want to use os.path.join() to construct a path to be used on a platform other than the one currently running the application. For example, a person might be running Python on Windows, but calling a web API and constructing a path for use on a remote posix system.
|
msg389493 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2021-03-25 00:32 |
The documentation of os.path.join() uses os.sep because it's written from the perspective of the platform's os.path, which is posixpath on a POSIX system and ntpath on a Windows system.
If you need to work with POSIX paths in Windows, then use posixpath. It will of course use '/' as the path separator, since using backslash (os.sep) in a POSIX path is wrong. Vice versa, if you need to work with Windows paths in POSIX, use ntpath.
|
msg389500 - (view) |
Author: Jared Sutton (jpsutton) * |
Date: 2021-03-25 13:48 |
I can understand your suggestion to just utilize the posixpath library on Windows if needed. That's a reasonable work-around. But certainly you can see this is a doc bug, since the doc clearly states that os.sep is utilized to join the elements of the path, when it clearly isn't; right?
|
msg389505 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2021-03-25 15:07 |
> certainly you can see this is a doc bug, since the doc clearly
> states that os.sep is utilized to join the elements of the path,
> when it clearly isn't; right?
os.path is ntpath in Windows, which uses backslash as the path separator, which is the same as os.sep in Windows. os.path is posixpath in POSIX, which uses slash as the path separator, which is the same as os.sep in POSIX.
|
msg389514 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-25 18:17 |
Perhaps Jared was expecting that modifying os.sep would affect the functions in os.path?
os.sep was never intended to be updated.
Using the specific *path modules to work with "foreign" paths has long been advocated as the way to do this. It isn't a work-around.
|
msg389521 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2021-03-25 19:20 |
Fred, do you think this needs to be reopened to clarify that os.sep and os.altsep are not parameters for the ntpath and posixpath modules?
I would have thought that the note at the top (path name conventions, etc) would be sufficient to let someone know that os.path is meant for working with paths on the current platform. Working with Windows paths in POSIX, and vice versa, is not simply a matter of replacing os.sep. For example:
>>> posixpath.join('C:/spam', '/eggs')
'/eggs'
>>> ntpath.join('C:/spam', '/eggs')
'C:/eggs'
|
msg389532 - (view) |
Author: Jared Sutton (jpsutton) * |
Date: 2021-03-26 01:03 |
> Perhaps Jared was expecting that modifying os.sep would affect the functions in os.path?
This is precisely what I thought, because the documentation makes it sound like that variable named os.sep is read and used as the path delimiter when constructing something with join(). In fact, that variable isn't read *at all* because the path separator is hard-coded in both posixpath.py and ntpath.py. Since os.sep isn't used, I see no reason why it should be referenced in the documentation at all.
I'm not trying to be pedantic here (though we nerds are famous for that :) ), but what I see here is a disagreement between what is documented and what actually exists in the implementation. Yes the *value* of os.sep happens to be the same as the one hard-coded into either ntpath or posixpath, but since that variable is not referenced in the implementation, its presence in the doc only serves to confuse people who take the documentation at face value and then get an unexpected result.
|
msg389539 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-26 04:51 |
Just reviewed the documentation for both os.sep and os.path.join().
The os.sep docs do not suggest it can be set, and the reference in the os.path.join() description is silent regarding that, so can be read as Jared did. I don't recall this coming up before, but... there's a lot I don't remember at this point. --sigh--
Removing the reference to os.sep from the os.path.join() description would not be confusing, IMO. Clearly, the ambiguity can bite.
I'd be open to a patch to remove the reference to os.sep from the os.path.join() docs. Reopening, but removing 3.6 and 3.7 since they're in security-fix-only mode.
|
msg389551 - (view) |
Author: Jared Sutton (jpsutton) * |
Date: 2021-03-26 14:13 |
Thank you for understanding my position, Fred. I submitted a draft PR (25025) for this bug.
|
msg389554 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-26 16:02 |
New changeset 21a2cabb3795f5170c746ab8f29e9d25c7442550 by Jared Sutton in branch 'master':
bpo-43620: Remove reference to os.sep from os.path.join() doc (#25025)
https://github.com/python/cpython/commit/21a2cabb3795f5170c746ab8f29e9d25c7442550
|
msg389555 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-26 17:24 |
New changeset f311290f091957653bba5ebfda28ad981bb78363 by Miss Islington (bot) in branch '3.9':
bpo-43620: Remove reference to os.sep from os.path.join() doc (GH-25025) (#25027)
https://github.com/python/cpython/commit/f311290f091957653bba5ebfda28ad981bb78363
|
msg389566 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-26 20:07 |
New changeset 455583b54aaec9a4266ff37dd438cbbd8ec6068a by Miss Islington (bot) in branch '3.8':
bpo-43620: Remove reference to os.sep from os.path.join() doc (GH-25025, GH-5030)
https://github.com/python/cpython/commit/455583b54aaec9a4266ff37dd438cbbd8ec6068a
|
msg389567 - (view) |
Author: Fred Drake (fdrake) |
Date: 2021-03-26 20:08 |
PR applied and backported; closing this.
Thanks, Jared!
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:43 | admin | set | github: 87786 |
2021-03-26 20:08:38 | fdrake | set | status: open -> closed resolution: fixed messages:
+ msg389567
stage: patch review -> resolved |
2021-03-26 20:07:07 | fdrake | set | messages:
+ msg389566 |
2021-03-26 19:52:30 | miss-islington | set | pull_requests:
+ pull_request23779 |
2021-03-26 17:24:41 | fdrake | set | messages:
+ msg389555 |
2021-03-26 16:03:30 | miss-islington | set | nosy:
+ miss-islington pull_requests:
+ pull_request23777
|
2021-03-26 16:02:42 | fdrake | set | messages:
+ msg389554 |
2021-03-26 14:13:47 | jpsutton | set | messages:
+ msg389551 |
2021-03-26 14:03:55 | jpsutton | set | keywords:
+ patch stage: needs patch -> patch review pull_requests:
+ pull_request23775 |
2021-03-26 04:51:41 | fdrake | set | status: closed -> open versions:
- Python 3.6, Python 3.7 messages:
+ msg389539
keywords:
+ easy resolution: not a bug -> (no value) stage: resolved -> needs patch |
2021-03-26 01:03:32 | jpsutton | set | messages:
+ msg389532 |
2021-03-25 19:20:59 | eryksun | set | messages:
+ msg389521 |
2021-03-25 18:17:21 | fdrake | set | nosy:
+ fdrake messages:
+ msg389514
|
2021-03-25 15:07:11 | eryksun | set | messages:
+ msg389505 |
2021-03-25 13:48:37 | jpsutton | set | messages:
+ msg389500 components:
- Library (Lib) |
2021-03-25 00:32:15 | eryksun | set | status: open -> closed
nosy:
+ eryksun messages:
+ msg389493
resolution: not a bug stage: resolved |
2021-03-24 20:35:52 | jpsutton | create | |