classification
Title: Blank lines in ._pth file are not ignored
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: Malcolm Smith, ammar2, excitoon, izbyshev, paul.moore, python-dev, r.david.murray, steve.dower, tim.golden, zach.ware
Priority: low Keywords: patch

Created on 2017-01-19 16:54 by steve.dower, last changed 2018-05-30 17:38 by excitoon. This issue is now closed.

Files
File name Uploaded Description Edit
empty_path.diff ammar2, 2017-01-21 01:48 review
underpth_blank_lines.diff izbyshev, 2017-01-27 15:04 review
Messages (16)
msg285809 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-01-19 16:54
If a python._pth file includes a blank line, it gets treated as '\n' which is then appended to the directory and used as an entry in sys.path.

Empty lines should be ignored completely.
msg285931 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2017-01-21 01:48
I've attached a patch that makes the site package ignore empty lines in .pth files. Along with a test for this behavior. I'm not really familiar with the site machinery so I hope the test and code are fine.
msg285960 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-01-21 16:21
The fix for this issue is needed in PC/getpathp.c as this is a special feature in path generation on Windows.

I'm not aware of any such issue with .pth files - the underscore in ._pth is deliberate.
msg286363 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2017-01-27 15:04
I've encountered this issue too. (FYI, the official 3.6.0 embeddable zip from https://www.python.org/downloads/windows/ does contain a blank line in its ._pth, so all its users get an invalid entry in sys.path).

The patch is attached. I couldn't fix tests in more straightforward way because both of them were broken:
* Both of them generated ._pth with 'import site'
* Both of them checked for zero exit code instead of non-zero one, but both passed because predicates in sys.exit() were incorrect

I've strengthened the check in 'nosite' test because it is trivial to fully emulate sys.path calculation logic in this case. I've preserved weaker checks in the other test because I didn't want to emulate site.py logic.
msg286432 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-01-29 06:30
I haven't had a chance to try the patch (underpth_blank_lines.diff) or apply it, but it looks right.

If one of the other core devs gets to apply this before I do, feel free.

Alexey, you don't need to sign a CLA for this fix, but if you're going to contribute more then you may want to: https://docs.python.org/devguide/committing.html#contributor-licensing-agreements It's all online, and it'll help save the core developers from having to evaluate whether you need it or not (which I can do, because it comes up often for me in my day job, but not everyone has that experience). And thanks for this patch!
msg287007 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-04 23:20
New changeset 54fea351e3f9 by Steve Dower in branch '3.6':
Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev)
https://hg.python.org/cpython/rev/54fea351e3f9

New changeset a0ff777ab153 by Steve Dower in branch 'default':
Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev)
https://hg.python.org/cpython/rev/a0ff777ab153
msg287013 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-05 00:00
New changeset 6d90b135e36c5b2794a29f650ebebf8bbd96f3fb by Steve Dower in branch 'master':
Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev)
https://github.com/python/cpython/commit/6d90b135e36c5b2794a29f650ebebf8bbd96f3fb

New changeset 3ba099c135cdbced6c2ff168a90a689a83b376db by Steve Dower in branch 'master':
Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev)
https://github.com/python/cpython/commit/3ba099c135cdbced6c2ff168a90a689a83b376db
msg287018 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-05 00:00
New changeset 6d90b135e36c5b2794a29f650ebebf8bbd96f3fb by Steve Dower in branch '3.6':
Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev)
https://github.com/python/cpython/commit/6d90b135e36c5b2794a29f650ebebf8bbd96f3fb
msg318033 - (view) Author: Malcolm Smith (Malcolm Smith) Date: 2018-05-29 14:08
> I'm not aware of any such issue with .pth files - the underscore in ._pth is deliberate.

An identical issue *does* exist for .pth files. As you can see from examining site.addpackage, it does not ignore blank lines as the documentation says. Ammar's patch should also be applied.
msg318126 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-05-29 22:28
Please create a new issue. This one is closed.
msg318182 - (view) Author: Vladimir Chebotarev (excitoon) * Date: 2018-05-30 13:19
Hi.

I guess this issue has to be reopen because Alexey erroneously broke adding empty paths to `sys.path` along with his patch.

Official embeddable Pythons do contain a blank line in its ._pth, and this allowed users to import modules from script directory.

Kind regards,
Vladimir.
msg318184 - (view) Author: Vladimir Chebotarev (excitoon) * Date: 2018-05-30 13:22
I shall attach pull request soon.
msg318196 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-05-30 16:05
The python._pth file generated for the embeddable distribution includes ".", which allows you to import from the dist directory. The blank line should be ignored.

If you want to misuse the embeddable distribution by not restricting its search paths, delete the ._pth file.
msg318198 - (view) Author: Vladimir Chebotarev (excitoon) * Date: 2018-05-30 16:48
Hi Steve.

I'll try to explain what is my motivation.

I need a reliable way to run Python (not matter embedded or not) in isolated mode, but still having current directory in `sys.path` (empty entry).

Ironically I could misuse normal mode to simulate isolated mode with specified %PYTHONPATH% (which is not much handy, by the way), but hiding Windows Registry keys would look pretty hard (it is impossible to make a bat/cmd wrapper). Possibility to search %PYTHONPATH% in HKLM entries makes it even worse.

Empty entry in `sys.path` is a extremely popular thing, and many software depend on it. It looks like a bug that isolated mode cannot support such entry.

# How would I fix it.

Possibly we should allow another keyword for adding an empty entry (like we have an exclusion for `import site`).

Ideas are appreciated.
msg318199 - (view) Author: Malcolm Smith (Malcolm Smith) Date: 2018-05-30 16:53
FYI: I have created issue 33689 for the non-Windows-specific issue.
msg318200 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-05-30 17:02
Vladimir, please open a new issue for this feature request.  This issue is closed.
History
Date User Action Args
2018-05-30 17:38:46excitoonsetpull_requests: - pull_request6869
2018-05-30 17:02:25r.david.murraysetnosy: + r.david.murray
messages: + msg318200
2018-05-30 16:53:20Malcolm Smithsetmessages: + msg318199
2018-05-30 16:48:38excitoonsetmessages: + msg318198
2018-05-30 16:05:22steve.dowersetmessages: + msg318196
2018-05-30 13:47:50excitoonsetpull_requests: + pull_request6869
2018-05-30 13:22:10excitoonsetmessages: + msg318184
2018-05-30 13:19:21excitoonsetnosy: + excitoon
messages: + msg318182
2018-05-29 22:28:54steve.dowersetmessages: + msg318126
2018-05-29 14:08:54Malcolm Smithsetnosy: + Malcolm Smith
messages: + msg318033
2017-02-08 05:06:13ammar2setstatus: open -> closed
2017-02-05 00:00:34python-devsetmessages: + msg287018
2017-02-05 00:00:31python-devsetmessages: + msg287013
2017-02-04 23:20:32steve.dowersetassignee: steve.dower
resolution: fixed
stage: needs patch -> resolved
2017-02-04 23:20:09python-devsetnosy: + python-dev
messages: + msg287007
2017-01-29 06:30:31steve.dowersetmessages: + msg286432
2017-01-27 15:04:28izbyshevsetfiles: + underpth_blank_lines.diff
nosy: + izbyshev
messages: + msg286363

2017-01-21 16:21:10steve.dowersetmessages: + msg285960
2017-01-21 01:48:11ammar2setfiles: + empty_path.diff

nosy: + ammar2
messages: + msg285931

keywords: + patch
2017-01-19 16:54:32steve.dowercreate