classification
Title: Tutorial/Modules - small fix to better clarify the modules search path
Type: Stage: resolved
Components: Documentation Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eric.araujo, eric.snow, ezio.melotti, georg.brandl, python-dev, r.david.murray, rhettinger, sandro.tosi, terry.reedy
Priority: low Keywords: patch

Created on 2011-04-28 17:18 by sandro.tosi, last changed 2012-01-19 21:03 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
tutorial_modules_actually.patch sandro.tosi, 2011-04-28 17:18
issue11948.patch sandro.tosi, 2011-04-29 20:04
issue11948-v2.patch sandro.tosi, 2011-05-06 23:20
issue11948-v3.patch sandro.tosi, 2011-05-09 17:54
issue11948-v4.patch sandro.tosi, 2011-05-09 19:05
issue11948-v5.patch sandro.tosi, 2011-05-11 19:58
Messages (30)
msg134704 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-04-28 17:18
Following up http://mail.python.org/pipermail/docs/2011-April/004029.html here's a small patch (applicable on all the active branches) to "actually" :) connects the two paragraphs about modules search path.
msg134721 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-04-28 21:50
It would be better to drop the introductory phrase entirely.
msg134809 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-04-29 20:03
Hi Raymond, thanks for looking into it! What do you think of this patch? I tried to save what I think was nice in the first paragraph and collapse it into the second one.
msg134822 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-04-29 23:28
I believe the patch produces the following as the first sentence

"When a module named :mod:`spam` is imported, the interpreter searches for a file named :file:`spam.py` in a list of directories given by the variable ``sys.path`` which is initialized from the directory containing the input script (or the current directory), :envvar:`PYTHONPATH` (a list of directory names, with the same syntax as the shell variable :envvar:`PATH`) and the installation-dependent default."

This is more that a 'mouthful', especially for a tutorial, and I think is should be split into two:
variable ``sys.path``.  ``Sys.path`` is initialized ...
msg134877 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-04-30 16:26
One caveat to Terry’s suggestion: never ever change the case of a Python identifier or keyword (“Sys”).
msg135384 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-05-06 23:20
Here's attached the patch including Terry's suggestion (gaah, sorry for this late reply).
msg135385 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-05-06 23:31
s/``sys.path``/:data:`sys.path`/ (or whatever turns that into a link).
I also don't like the "Python programs that know what they're doing" bit.
The list of places where the modules are searched could be a bullet list.
msg135620 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-05-09 17:54
What about the attached (v3) patch? the "This allows...experts hands." part can be remove completely if can be mis-interpreted.
msg135627 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-09 18:55
Please don't make these kind of changes.  Guido has been clear that the docs need to affirmatively state what the language does.  

Except for certain security risks or segfault risks, the docs should avoid wording along the lines of "feature x is dangerous" or "experts only".  These kind of value judgements belong in blogs and wikis, not in the core documentation which should simply focus on what the language does and examples of how to use it effectively.
msg135629 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-05-09 19:05
Sorry Raymond for the bad editing, I was confused by thinking 'tutorial' is particularly meant for new-coming people. Anyhow, sorry again: I attach a patch removing the badworded part.

Please let me know if anything else is missing or needs fixing.

Thanks for your check & time.
msg135630 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-05-09 19:08
Raymond: I haven't read the examples in detail so I don't have an opinion on the appropriateness of the text, but I'm curious: while what you say certainly applies to the Language Reference, does it also apply to the Tutorial?
msg135632 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-09 19:12
The tutorial is meant for newcomers.  The experience needs to be positive and not leave the reader with worries that something bad will happen if they make a misstep.

People commonly use the tutorial to evaluate the language as a whole.  The tutorial needs to show people what the language is capable of doing.  It should not create the impression that all its features are dangerous and should only be used by experts.
msg135634 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-09 19:19
One other stylistic note.  The tone of the tutorial (and all the docs) needs to be respectful of the readers intelligence.  We don't presume that the readers are stupid.  We lay out the relevant information, show motivating use cases, provide glossary links, and do our best to connect-the-dots, but don't talk down them or waste their time.

The recent doc patch for using the in-operator instead of str.find borders on the edge of not giving readers credit for being able to make trivial inferences about the API.
msg135656 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-05-09 21:39
Raymond, thanks for explaining the reasoning behind your replies, it really helps me understand the "spirit" behind python :)

That said, and trying to be a bit more pragmatical, what are the next steps to make this patch acceptable? It's kinda of a small patch, and I honestly didn't foresee the long ping-pong of corrections I received :)
msg135662 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-09 22:41
Sandro, import is a somewhat dark corner of Python that newcomers and even experienced people ofter trip over. So getting the text both correct and clear is worth some thought.

I just noticed that the text is only true for Python-coded modules and not for built-in modules. I discovered this by setting sys.path to {} and importing itertools. It works. It appears that Python first searches for builtins -- I created itertools.py and it did not mask the builtin. If there were a Lib/itertools.py which imported _itertools, it would have been masked. So it appears that the text should begin

"When a module named :mod:`spam` is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file ..."

I suggest changing
"This allows Python programs to modify or replace the module search path."
to
"After initialization, Python programs can modify sys.path."

and changing the followup

" Note that because the directory containing the script being run is on the search path, it is important that the script not have the same name as a standard module, or Python will attempt to load the script as a module when that module is imported.  This will generally be an error."

to something more like

"Because the directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path, the script and other scripts in that directory should not have the same name as a standard library module unless one really intends for the script to be loaded in place of the standard library module."

or perhaps

"The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended."

I think the change to warn about all possibly conflicting scripts in the directory is needed because a common problem people have is more like

import random
a = random.choice('abc')

raising AttributeError ("O python-list, why???") because there is a forgotten random.py in the same directory.
msg135667 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-05-09 23:52
Raymond: yes, thanks for the clear guidance.  Perhaps it could be added to Documenting Python, perhaps in the Style Guide chapter?
msg135673 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-10 01:12
I agree with both Raymond and David's suggestion.
msg135676 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-05-10 04:46
+1 to the guidance, and to adding it to the doc style chapter.
msg135680 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-10 06:58
See http://bugs.python.org/issue12047
msg135798 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-05-11 19:58
I first would like to apologize if my comments were interpreted like I'd like to do a poor quality job, that's actually the opposite! I just get caught by surprise how a simple change in a word to better join two paragraphs grows up like this - but that's perfectly fine and understandable we want to be as detailed and precise as possible

Here's the fifth version of the patch, including Terry's suggestions - thanks!
msg135932 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-13 17:22
About Terry’s suggestion to change "This allows Python programs to modify or replace the module search path" to "After initialization, Python programs can modify sys.path": IMO the old text is talking about intent (modify the module search path) and the replacement about implementation (modify sys.path).  I prefer the former; what do you think?
msg135937 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-13 18:10
I disagree (else I would not have suggested change ;-). First, I dislike 'This allows' on stylistic grounds, when there is a better alternative. It is rather wishy-washy: 'this allows' -- so what? As for the rest -- why not be specific? The whole paragraph is about sys.path, so it is not an hidden detail.
msg135940 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-13 18:17
> First, I dislike 'This allows' on stylistic grounds
Fully agreed, sir!

> The whole paragraph is about sys.path, so it is not an hidden detail.
It may be that I miss context; I don’t actually know whether the paragraph is only about sys.path or about module search in general.  Apologies if my comments are only noise.  This is my last proposal:

“After initialization, Python programs can modify sys.path to modify or replace the module search path (except for built-in modules)”

If it’s redundant, then +1 to your earlier suggestion.
msg135942 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-13 18:22
Sandro, thank you for sticking with this. Seemingly simples issues sometimes 'explode' a bit. Having reviewed the patch, I think the it is ready to be committed.

Éric: if you were to commit this and, in the process, wanted to change sys.path back to 'this module search path' ('this' instead of 'the' to refer back to the definition of sys.path just given), I would not object.
msg137069 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-27 15:53
I think I’d better leave this one to Terry or Raymond.
msg150993 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-01-10 00:48
Sandro, I think you can apply this.
msg151613 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-01-19 10:32
New changeset d01208ba482f by Sandro Tosi in branch '2.7':
Issue #11948: clarify modules search path
http://hg.python.org/cpython/rev/d01208ba482f

New changeset 6d663db63705 by Sandro Tosi in branch '3.2':
Issue #11948: clarify modules search path
http://hg.python.org/cpython/rev/6d663db63705

New changeset 93769b8ff40b by Sandro Tosi in branch 'default':
Issue #11948: merge with 3.2
http://hg.python.org/cpython/rev/93769b8ff40b
msg151614 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2012-01-19 10:34
Thanks Terry for the ping, I've just committed it - thanks again to everyone for the help/suggestions.
msg151643 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-01-19 16:52
FYI: unless importlib took undue liberties (unlikely), frozen modules also precede path-based modules.  See the implicit additions to sys.meta_path in Lib/importlib/_bootstrap.py.

Whether or not to include a mention of frozen modules in the tutorial...I'll leave that to you.  :)
msg151653 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-01-19 21:03
Definitely out-of-scope for the tutorial. I consider this akin to monkey patching imported modules.
History
Date User Action Args
2012-01-19 21:03:05terry.reedysetmessages: + msg151653
2012-01-19 16:52:01eric.snowsetnosy: + eric.snow
messages: + msg151643
2012-01-19 10:34:16sandro.tosisetstatus: open -> closed
resolution: fixed
messages: + msg151614

stage: commit review -> resolved
2012-01-19 10:32:52python-devsetnosy: + python-dev
messages: + msg151613
2012-01-10 00:48:42terry.reedysetmessages: + msg150993
versions: - Python 3.1
2011-05-27 15:53:15eric.araujosetmessages: + msg137069
2011-05-13 18:22:55terry.reedysetstage: patch review -> commit review
2011-05-13 18:22:26terry.reedysetmessages: + msg135942
2011-05-13 18:17:24eric.araujosetmessages: + msg135940
2011-05-13 18:10:15terry.reedysetmessages: + msg135937
2011-05-13 17:22:21eric.araujosetmessages: + msg135932
2011-05-11 19:58:22sandro.tosisetfiles: + issue11948-v5.patch

messages: + msg135798
2011-05-10 06:58:03rhettingersetmessages: + msg135680
2011-05-10 04:46:51georg.brandlsetnosy: + georg.brandl
messages: + msg135676
2011-05-10 01:12:34terry.reedysetmessages: + msg135673
2011-05-09 23:52:31r.david.murraysetmessages: + msg135667
2011-05-09 22:41:13terry.reedysetmessages: + msg135662
2011-05-09 21:39:22sandro.tosisetmessages: + msg135656
2011-05-09 19:19:31rhettingersetmessages: + msg135634
2011-05-09 19:12:59rhettingersetmessages: + msg135632
2011-05-09 19:08:52r.david.murraysetnosy: + r.david.murray
messages: + msg135630
2011-05-09 19:05:11sandro.tosisetfiles: + issue11948-v4.patch

messages: + msg135629
2011-05-09 18:55:29rhettingersetmessages: + msg135627
2011-05-09 17:54:09sandro.tosisetfiles: + issue11948-v3.patch

messages: + msg135620
2011-05-06 23:31:54ezio.melottisetnosy: + ezio.melotti
messages: + msg135385
2011-05-06 23:20:30sandro.tosisetfiles: + issue11948-v2.patch

messages: + msg135384
2011-04-30 16:26:59eric.araujosetnosy: + eric.araujo
messages: + msg134877
2011-04-29 23:28:46terry.reedysetnosy: + terry.reedy
messages: + msg134822
2011-04-29 20:04:16sandro.tosisetfiles: + issue11948.patch
2011-04-29 20:03:48sandro.tosisetmessages: + msg134809
2011-04-28 21:50:53rhettingersetpriority: normal -> low
nosy: + rhettinger
messages: + msg134721

2011-04-28 17:18:30sandro.tosicreate