classification
Title: add support for path-like objects in sys.path
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, chris.jerdonek, jayyin11043, yan12125
Priority: normal Keywords: patch

Created on 2018-01-24 05:27 by chris.jerdonek, last changed 2018-03-18 22:18 by jayyin11043.

Pull Requests
URL Status Linked Edit
PR 5691 open jayyin11043, 2018-02-15 05:42
Messages (16)
msg310560 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2018-01-24 05:27
This issue is to suggest enhancing sys.path to recognize path-like objects, per PEP 519.

I recently ran into an issue where a path was getting added to sys.path, but the corresponding imports weren't working as they should, even though sys.path showed the path as being present. Eventually I tracked this down to the path being a pathlib.Path object rather than a string. This wasn't obvious because Path objects appear as strings in normal debug output, etc.

The sys.path docs currently say this:

> Only strings and bytes should be added to sys.path; all other data types are ignored during import.
msg311693 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-02-05 21:23
This looks a lot like https://bugs.python.org/issue32446, I'd like to tackle this, if we are going through with it.
msg311864 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-02-09 03:49
what file(s) is/are the sys.path code located in?
msg311895 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2018-02-09 16:11
> what file(s) is/are the sys.path code located in?

If I understand it correctly, sys.path is handled in importlib._bootstrap_external.PathFinder.find_spec(). I can patch PathFinder for handling path-like objects: https://github.com/yan12125/cpython/commit/e3fd473b54cbb368533e651fd896bbc813a43924

Here's an example usage:

# t.py
import pathlib
import sys

sys.path.append(pathlib.Path('foo'))

import s

# foo/s.py
print(123)
msg311986 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-02-11 08:49
Thanks for the reply
msg312070 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-02-12 19:44
ok, so I found the PathFinder class you referenced, just making sure, this issue pertains to changing "self.path"'s usage and declaration to be a path-like object instead of the hard coded 'sys', 'path' returns? or is that part of it already?, also since this uses a tuple to keep track of the parent path, is there functionality in the path-like objects to find parent paths?
msg312072 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-02-12 19:51
https://github.com/python/cpython/blob/3c34aad4e7a95913ec7db8e5e948a8fc69047bf7/Lib/importlib/_bootstrap_external.py#L1069-L1090

those are the particular class and lines I'm referring to
msg313283 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-05 20:43
I'm unsure how to regenerate the files that interact with the code for sys.path as travisCI states
"
Generated files not up to date
 M Python/importlib_external.h
"
since my code changes some of how the importing is handled
msg313337 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-03-06 15:53
'make regen-all' should do it (but so will just running 'make -s -j' like
any old build of CPython).

On Mon, Mar 5, 2018, 12:43 Jay Yin, <report@bugs.python.org> wrote:

>
> Jay Yin <jayyin11043@gmail.com> added the comment:
>
> I'm unsure how to regenerate the files that interact with the code for
> sys.path as travisCI states
> "
> Generated files not up to date
>  M Python/importlib_external.h
> "
> since my code changes some of how the importing is handled
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32642>
> _______________________________________
>
msg313388 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-07 15:17
I've been stuck on "test_poplib" for over 149661 sec now, that's about 41 hours... I don't think this is working correctly, although I'm unsure  what test_poplib is doing that has been affected by what I've changed here.
msg313460 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-08 22:00
The issue was resolved by updating my version of the rest of the package apparently and remaking the whole thing, must have been some outdated stuff on my end causing the issue
msg313883 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-15 14:13
I'm having issues with my local changes for my PR, I'm unsure why my local machine takes a seemingly infinite amount of time on test_poplib, so again I think something to do with my local environment is causing issues again, any help would be appreciated...
msg313885 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-15 14:29
https://pastebin.com/q4FKnPZH

the trace for the test_poplib
msg313887 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-15 15:32
it seems to me like the issue in my tests is that some SSL thing is failing?, anyone have any experience with this?
msg313923 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-03-15 23:51
Please open a separate issue for the test_poplib issue.
msg314065 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-18 22:18
srry I opened another issue bpo-33099
History
Date User Action Args
2018-03-18 22:18:44jayyin11043setmessages: + msg314065
2018-03-15 23:51:48brett.cannonsetmessages: + msg313923
2018-03-15 15:32:45jayyin11043setmessages: + msg313887
2018-03-15 14:29:54jayyin11043setmessages: + msg313885
2018-03-15 14:13:03jayyin11043setmessages: + msg313883
2018-03-08 22:00:59jayyin11043setmessages: + msg313460
2018-03-07 15:17:49jayyin11043setmessages: + msg313388
2018-03-06 15:53:30brett.cannonsetmessages: + msg313337
2018-03-05 20:43:42jayyin11043setmessages: + msg313283
2018-02-15 05:42:58jayyin11043setkeywords: + patch
stage: patch review
pull_requests: + pull_request5484
2018-02-12 19:51:22jayyin11043setmessages: + msg312072
2018-02-12 19:44:51jayyin11043setmessages: + msg312070
2018-02-11 08:49:54jayyin11043setmessages: + msg311986
2018-02-09 16:11:11yan12125setnosy: + yan12125
messages: + msg311895
2018-02-09 03:49:23jayyin11043setmessages: + msg311864
2018-02-05 21:23:01jayyin11043setnosy: + jayyin11043
messages: + msg311693
2018-01-24 05:27:12chris.jerdonekcreate