New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can not import modules if sys.prefix contains DELIM #75393
Comments
If the path on which Python is installed contains the DELIM, the resulted sys.path is split. |
By DELIM, you mean the shell ':'? As far as I've been able to determine there is no way defined in posix to escape that character. If you can find one, please let us know. (I think the same is true for the Windows semicolon but I'm not sure.) |
Yes I mean ':' for posix. |
I'm not sure there is anything we should do here, then, because we are conforming to the posix parsing for PATH in our PYTHONPATH implementation. I think if you want to pursue this further you should take it to the python-ideas mailing list. I'm going to close the issue, but you can reopen it if python-ideas thinks it is worth doing something (and there is something we can reasonably do :) |
A last comment, I do not think it is an issue to follow posix way to parse PATH. But for me, the problem is that Python adds without sanitation the sys.prefix to the PYTHONPATH. So I think internally Python should not use PATH notation to extend the PYTHONPATH because it is not a robust notation. |
You mean to create the entries on sys.path that do not come from the PYTHONPATH? |
On 2017-08-15 17:32, R. David Murray wrote:
Yes because such path could contain ':'. |
OK, that seems reasonable to me. I'll reopen the issue. Assuming other developers agree that this should be changed, I'm not sure if it will qualify as a bug or an enhancement, so I'm leaving versions unselected for now :) |
I'm marking this as Python 3.7 only, not because I don't think it's a bug, but because getpath.c is a maintainability nightmare and I strongly prefer we avoid going anywhere near it in maintenance releases :) Targeting Python 3.7+ also means we may be able to take advantage of the initial phase of PEP-432 changes and look at switching getpath.c to use higher level CPython data structures (specifically list objects) internally. I've also added the Windows folks to the nosy list to check if Windows might have a similar problem with semi-colons appearing in sys.prefix. |
We don't. Semicolons are not valid path characters, so you can't have them in a single path (unless you're deliberately trying to break your entire machine...) |
Is this post wrong then?: ("I noticed that the semicolon ; is a valid character for Windows (NTFS) file and directory names.") |
No, it's not wrong that semicolon is a valid character in Windows NTFS and FAT32 filesystems. However, the answer by Kevin Edwards that recommends using double quotes needs qualification. It only works in CMD. It doesn't work in PowerShell nor the runtime library functions used by CreateProcess or SearchPath (i.e. RtlDosSearchPath_U[str]) or the loader's LdrpSearchPath function. These all split PATH on semicolon, with no supported escape character or quoting. In fact they retain double quotes in the computed filename when testing for existence, so quoting paths in PATH is wrong in general. |
All right. So the challenge here for windows is: if python is installed on a path that has a semicolon in one of the directory names, is it even possible to populate sys.path such that modules can be imported from the lib directory that is under that path? (IMO we shouldn't jump through big hoops to support this, but if it can be done and the refactoring Nick is contemplating makes it *relatively* easily, it would probably be good to do, to be consistent). |
It's simple to work around using a junction or symbolic link, so is this worth pursuing? |
I'm wondering if it could have security implications and be used to fool user by changing the PYTHONPATH. |
If you have access to modify PYTHONPATH at all, you can already shadow almost all standard library modules: $ PYTHONPATH=/MY_CHOSEN_DIRECTORY python3 -m site
sys.path = [
'/home/ncoghlan',
'/MY_CHOSEN_DIRECTORY',
'/usr/lib64/python36.zip',
'/usr/lib64/python3.6',
'/usr/lib64/python3.6/lib-dynload',
'/home/ncoghlan/.local/lib/python3.6/site-packages',
'/usr/lib64/python3.6/site-packages',
'/usr/lib/python3.6/site-packages',
] The only ones you can't shadow that way are builtin and frozen modules, and any modules that get imported even before PYTHONPATH is processed. So no, this doesn't open up any new attack vectors that weren't already present by design. As far as whether or not it's worth fixing goes, yes, I think so - one of my original motivations for writing PEP-432 was to allow the use of CPython data structures when calculating the initial value of sys.path, and this is a nice concrete example of a bug arising from the current implementation. |
I consider that the current behavior is correct. They are other ways to put a path which contains ":" into sys.path. I close the issue. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: