This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: sys.argv[0] and python -m package
Type: behavior Stage: resolved
Components: Versions: Python 3.2
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: benjamin.peterson, brett.cannon, michael.foord, ncoghlan
Priority: normal Keywords:

Created on 2010-03-22 20:21 by michael.foord, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (11)
msg101531 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-22 20:21
When you execute "python -m package" the package is first imported with sys.argv[0] set to '-c' (and sys.modules['__main__'] exists but is empty. Then package.__main__.py is executed with the correct sys.argv[0].

This means module level code executed during the initial import that attempts to work out how the code is being executed can't use either sys.modules['__main__'] *or* sys.argv[0].
msg101538 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-03-22 22:05
Hmm, the problem here is we don't *know* the module filename until we get hold of the loader for it (see _get_module_details() in runpy). And it is the process of finding the loader for the __main__ module that does the initial import of the package as a side effect.

Added Brett to the nosy list in case I've not thought of something obvious.
msg101542 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2010-03-22 22:43
Nick is right that importing package.__main__ requires importing package.__init__ first.

But it sounds like Michael really just wants some way to know when runpy is being used over something else. Could a special string token like "<runpy>" or "<-m>" be placed in sys.argv until it is back-patched to what it will eventually become? That should allow Michael to find out what whether runpy was being used or not.
msg101544 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-22 22:45
I've already fixed my specific usecase (for unittest), but a special value in sys.argv[0] would indeed have met my needs.
msg101546 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-22 22:48
'-c' isn't helpful because that is also the content of sys.argv[0] when a module is imported in response to "python -c '...'".
msg101582 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-03-23 14:27
I realised today that this behaviour is actually the case for any execution of a module inside a package with -m (i.e. any __init__ modules execute before sys.argv and __main__ are fully configured).

As I recall, I used a bit of a hack to get main.c to implement -m correctly by piggybacking on the existing -c semantics. I'll find the hack and replace it with some proper '-c' or '-m' logic.
msg104434 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-04-28 14:58
Committed for 2.7 in r80580 (as it turns out, the hack I remembered was probably from the original pre-runpy 2.4 implementation and has since been replaced by the proper runpy based system. This bug was likely just a lingering remnant of that original hackish approach).

Will forward port to 3.2 after the 2.7 beta is out.
msg107651 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-06-12 13:14
Grr, in doing the Py3k forward port I found the hack based on the "-c" entry that I remembered (I had mentioned it *right there* in a comment in main.c, so I don't know how I missed it when first updating 2.7 - I suspect I got lost in the ifdef maze inside SetArgV and managed to convince myself it wasn't a problem).

Anyway, it turns out PySys_SetArgV() uses the "sys.argv[0] == '-c'" check to skip over checking the file system for a file by that name. By reusing that same value, the -m code was able to also skip that check without needing to add any additional checks in the sysmodule code.

With the change I made to the 2.x branch, 2.7 now reacts differently if someone creates a "-m" file in the launch directory:

:~/devel/python$ python -i -m runpy
No module specified for execution
>>> sys.path[0]
''
:~/devel/python$ ./python -i -m runpy
No module specified for execution
>>> sys.path[0]
''
>>> 
:~/devel/python$ cat > -m
dd
:~/devel/python$ cat < -m
dd
:~/devel/python$ python -i -m runpy
No module specified for execution
>>> sys.path[0]
''
:~/devel/python$ ./python -i -m runpy
No module specified for execution
>>> sys.path[0]
'/home/ncoghlan/devel/python'
>>> 

System python is 2.6, local Python is SVN head. Note that the version from SVN changes behaviour after I create the oddly named file, while the system Python is unaffected.

My inclination is to fix this properly for 3.2 (including some extra command line tests to ensure that files named "-c" and "-m" don't confuse the sys.path configuration), but revert the change completely for 2.7.

Added Benjamin for an opinion, since 2.7 is in RC mode already.
msg107720 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-06-13 02:44
Revert seems sensible in this situation.
msg107723 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-06-13 06:55
Reverted in r81964, marking as 3.2 only.

For 3.2, I'll get rid of the hack by having SetArgV treat both "-c" and "-m" as special values for sys.argv[0], and by adding some unit tests that make sure the presence of files with those names has no effect on the value of sys.path[0] when the corresponding command line switches are used.
msg114116 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-08-17 13:08
Implemented in r84140.

Leaving issue open until the buildbots give it a clean bill of health (since the command line tests can be a little quirky when it comes to cross platform differences)
History
Date User Action Args
2022-04-11 14:56:58adminsetgithub: 52449
2010-10-24 09:22:15ncoghlansetstatus: open -> closed
2010-08-17 13:08:28ncoghlansetresolution: accepted
messages: + msg114116
stage: test needed -> resolved
2010-06-13 06:55:42ncoghlansetstage: commit review -> test needed
messages: + msg107723
versions: - Python 2.7
2010-06-13 02:44:32benjamin.petersonsetmessages: + msg107720
2010-06-12 13:14:25ncoghlansetstatus: pending -> open

nosy: + benjamin.peterson
messages: + msg107651

resolution: accepted -> (no value)
stage: resolved -> commit review
2010-04-28 14:58:13ncoghlansetstatus: open -> pending
resolution: accepted
messages: + msg104434

stage: resolved
2010-03-23 14:27:34ncoghlansetmessages: + msg101582
2010-03-22 22:48:32michael.foordsetmessages: + msg101546
2010-03-22 22:45:54michael.foordsetmessages: + msg101544
2010-03-22 22:43:52brett.cannonsetmessages: + msg101542
2010-03-22 22:05:35ncoghlansetnosy: + brett.cannon
messages: + msg101538
2010-03-22 20:21:44michael.foordcreate