classification
Title: help("modules") executes module code
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: devplayer, docs@python, dronus, eric.araujo, ezio.melotti, rhettinger, terry.reedy
Priority: normal Keywords:

Created on 2011-09-05 19:31 by dronus, last changed 2014-07-11 03:46 by devplayer.

Messages (10)
msg143556 - (view) Author: (dronus) Date: 2011-09-05 19:31
When running help("modules"), some code of the modules seems to be run. For example, on my Gnome system KWallet  configuration GUI pops up and stalls further opereation. I think no one expect help("modules") run abitrary code instead of just outputting a list of modules.
msg143616 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-06 15:35
Thanks for the report.  This comes from the fact that pydoc imports the modules in order to get their documentation.  Your message makes me think that the KWallet Python modules have a problem: they should not pop GUIs at import time!  Please report this to them.

In the Python docs, there is a note warning about this: http://docs.python.org/dev/library/pydoc .  The doc for help, which is not on the same page, does not: http://docs.python.org/dev/library/functions#help

It could be possible to load the code of a module without executing it.  I’m not sure it would be a good idea; maybe you could ask for opinions on the python-ideas mailing list?
msg143786 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-09-09 18:37
The immediate issue is improvement of the entries for help and help():

In builtin functions section:

Expand "Invoke the built-in help system." to
"Invoke the built-in help system, which uses *pydoc*."
where *pydoc* is linked to the pydoc section.

Add to the end of the entry something like "As needed, pydoc imports modules to get their doc strings."

The special string 'modules' is not documented in the manual entry for help() itself. That is fine. It does appear in the longer text that appears for 'help()'. That text, where ever it is, could and should have a warning.

"Warning: gathering the results for the topic 'modules' can take considerable time and have side effects as it imports *every* module available to get at its doc string."

A separate issue would be a feature request to not do that (assuming it really does). Given that 'modules' just produces a list of (incomplete) module names, I do not see the point of importing until one requests help on a particular name.

There seems to be a bug. In a fresh 3.2.2 IDLE window, help('modules') produces about 300 names. If I repeat, I get a reduced list of 58 names. These seem to just be the builtin C-coded modules, like 'math', with all Python-coded modules, like 'random', omitted.
msg143829 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-09-10 05:04
> "Warning: gathering the results for the topic 'modules' can take
> considerable time and have side effects as it imports *every* module
> available to get at its doc string."

http://docs.python.org/documenting/style.html#affirmative-tone
msg144414 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-09-22 19:00
Since help("modules") just shows a list of modules without any description, maybe it could avoid importing them until further information about individual modules are requested.

Also while trying to do help("modules") twice on py3k, I noticed that the second time many modules are missing from the list.  Is this expected and/or a known issue?
msg144424 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-09-22 21:57
As I said in msg143786, the second time only c-coded modules are listed.
msg144425 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-09-22 22:03
Right, I missed that.
If we change the way the list is created this bug will be probably get fixed too.  If we don't, we should open another issue.
msg144432 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-09-22 23:40
> A separate issue would be a feature request to not do that (assuming it really does).

I tried to verify this (the fact that modules get imported with help('modules')).  Here are the results:
  * Lib/foobar.py:
    - help('modules'): foobar in the list, code is *not* executed;
    - help('foobar'): code is executed;
  * Lib/deleteme/__init__.py:
    - help('modules'): deleteme in the list, code is executed;
    - help('deleteme'): code is executed;
  * Lib/deleteme/__init__.py and Lib/deleteme/foobar.py:
    - help('modules'): deleteme in the list, only __init__ code is executed;
    - help('deleteme.foobar'): foobar code is executed;

So it seems the only code that gets executed with help('modules') is the one in packages' __init__s.  It also seems that the code is executed only once, so doing help('deleteme') after help('modules') doesn't execute the code again.
msg152487 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-02-03 00:07
#13902 is essentially a duplicate of this and I may close it.
I am thinking now that executing unknown amounts of unknown code from unknown modules is a really bad idea. If a module just crashes the system, as happened with the OP of the above, there is no way to tell what the culprit is. So if we do not disable 'modules', I thing it should just read directory names and forget about docstrings. In any case, output should be flushed as available if not now.
msg222721 - (view) Author: Dev Player (devplayer) Date: 2014-07-11 03:46
Mentioned for informational purposes only.

I too experience the running of external packages with a different library when doing help('modules') in the interpreter. This is a fresh install of Python 3.4 on WinXP.

The text I get in the python.exe interpreter is:
"Expected Tk Togl installation in C:\Python\Python34\lib\site-packages\OpenGl\Tk\togl-win32"

Then I get an empty TK popup window.

Although this other issue was ages ago and was with a different machine, Python version and set of libraries I'm giving reference to this only because of mention of help(). http://bugs.python.org/issue10060

Although there is a command line option to prevent the import of site, while I may not want "help" to be imported, I usually want "site" to be imported. It would be nice to exclude the "help" import only via a command line.

I wonder if the interpreter could be given a command line option just to parse modules/scripts/packages/librarys to only compile the lines containing def and class without anything within the namespace except implicitly declared docstrings  (it not __doc__ = """...""")

In other words if you had source like:
def somefunc(arg=None):
   """here is the func __doc__"""
   x = value
   callme()

the interpreter could basically compile that into:

def somefunc(arg-None):
    """here is the func __doc__"""
    return None

or perhaps shortcircuit any non def/class/ """ """ to be tokenized as the pass statement would be.

Those would be feature/enhance kind thing I suppose.
History
Date User Action Args
2014-07-11 03:46:15devplayersetnosy: + devplayer
messages: + msg222721
2012-02-04 00:39:58terry.reedylinkissue13926 superseder
2012-02-03 00:07:46terry.reedysetmessages: + msg152487
2011-09-22 23:40:14ezio.melottisetmessages: + msg144432
2011-09-22 22:03:15ezio.melottisetmessages: + msg144425
2011-09-22 21:57:32terry.reedysetmessages: + msg144424
2011-09-22 19:00:23ezio.melottisetnosy: + rhettinger
messages: + msg144414
2011-09-10 05:04:28ezio.melottisetmessages: + msg143829
2011-09-09 18:37:37terry.reedysetassignee: docs@python
components: + Documentation
versions: + Python 2.7, Python 3.2, Python 3.3
nosy: + terry.reedy, docs@python

messages: + msg143786
stage: needs patch
2011-09-08 10:22:45ezio.melottisetnosy: + ezio.melotti
2011-09-06 15:35:37eric.araujosetversions: - Python 2.7
nosy: + eric.araujo

messages: + msg143616

components: - None
2011-09-05 19:31:22dronuscreate