This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Tutorial doesn't discourage name mangling
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: docs@python, eric.araujo, ezio.melotti, georg.brandl, lukasz.langa, python-dev, rhettinger, sheep, tim.golden
Priority: normal Keywords:

Created on 2011-05-16 08:35 by sheep, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (12)
msg136072 - (view) Author: Radomir Dopieralski (sheep) Date: 2011-05-16 08:35
In the tutorial, at http://docs.python.org/tutorial/classes.html#private-variables you can read:

----
9.6. Private Variables
“Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.

Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclasses), there is limited support for such a mechanism, called name mangling. Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.

Note that the mangling rules are designed mostly to avoid accidents; it still is possible to access or modify a variable that is considered private. This can even be useful in special circumstances, such as in the debugger.

[...]
----

I think that this section doesn't stress enough how special the "__foo" syntax is and how rarely it should be used. If I was a programmer coming from Java to Python, I would start using "__foo" everywhere after reading this. I actually receive code written like that from programmers new to Python, and they point to that section of documentation when I ask why they did it.

At minimum, I'd add a paragraph that warns about how name mangling makes the code hard to reuse, difficult to test and unpleasant to debug.
msg136073 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-05-16 08:42
No warnings, please.  See http://docs.python.org/documenting/style.html#affirmative-tone and related sections.
msg136074 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-05-16 08:52
But at the least, the start of the para might be slightly reworded to something like: "If you specifically need to avoid name clashes with subclasses, there is limited support..." which avoids the phrase "Since there is a valid use-case for class-private members".
msg136075 - (view) Author: Radomir Dopieralski (sheep) Date: 2011-05-16 08:54
"In the unlikely case that you specifically need to avoid name clashes with subclasses, there is limited support..." ;)
msg136076 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-05-16 09:34
I think the main problem here is that they are called "private" and people coming from other languages assume that the double underscore is the Python way to mark variables as private, whereas the appearance of being private is just a side effect of the name mangling.

This part should IMHO focus on the name mangling, explaining what it is and what it's useful for, and mention clearly that this doesn't really make the variable "private", because it can still be accessed through the mangled name.

The section could also mention that the word "private" here doesn't mean the same thing as in other language.

I don't know if the "trust the programmer" philosophy found in Python is explained somewhere in the doc, but if not it could be added to the tutorial or the FAQ.
msg136077 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2011-05-16 10:17
I'd like to see that section gone altogether. The tutorial is designed to be the first point of contact with the language. In that context name mangling is an obscure detail with an unlikely use case.

If anything, the tutorial should simply state that private variables are by convention prepended with a single underscore. Maybe there's place for a separate short article which would nicely explain the philosophy behind the lack of an enforcing `private` modifier. The "we're all consenting adults here" stuff.

Agreed with Ezio that "private" is in itself a poor name for what underscore (regardless whether a single one or two) really means.
msg136133 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-17 02:06
This is a venerable section of the tutorial that has been exposed to many, many readers and AFAICT it has never caused a problem.  So, I see no need to make a change to it.

Also as Georg points out, the docs (especially the tutorial) are not about discouragement, they are about empowerment.
msg136145 - (view) Author: Radomir Dopieralski (sheep) Date: 2011-05-17 09:16
I am reporting this specifically because I just had two independent cases of people who submitted code that had almost all methods name-mangled (within 2 weeks), and who then pointed to that section of the tutorial as justification. I have a hard time convincing them that it is a bad idea, as I have to work against the official documentation here. 

I agree that the language and library references should explain the mechanics behind the language in a neutral and empowering way. But I think that tutorial shouldn't tell people to write horrible code.

Perhaps it would suffice if the tutorial didn't call this "private methods"? A more descriptive and accurate section name, such as "name mangling" or "avoiding name clashes" could help a lot.
msg136146 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-05-17 10:33
FWIW that section used to be called "Private variables through name mangling" back in 1.5, and started by saying "There is now limited support for class-private identifiers.".

PEP8 [0] also talks about the name mangling in several places, and carefully avoids the use of "private":
  """
  To avoid name clashes with subclasses, use two leading underscores to
  invoke Python's name mangling rules.

  Python mangles these names with the class name: if class Foo has an
  attribute named __a, it cannot be accessed by Foo.__a.  (An insistent
  user could still gain access by calling Foo._Foo__a.) Generally, double
  leading underscores should be used only to avoid name conflicts with
  attributes in classes designed to be subclassed.
  """

And it even includes the following two notes:
  """
  We don't use the term "private" here, since no attribute is really
  private in Python (without a generally unnecessary amount of work).
  """

  """
  Not everyone likes name mangling.  Try to balance the
  need to avoid accidental name clashes with potential use by
  advanced callers.
  """

[0]: http://www.python.org/dev/peps/pep-0008/
msg139071 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-25 14:28
New changeset 68bc3c5960a4 by Raymond Hettinger in branch '2.7':
Issue 12086: add example showing how to use name mangling.
http://hg.python.org/cpython/rev/68bc3c5960a4
msg139072 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-25 14:31
New changeset 79f9698777b7 by Raymond Hettinger in branch '3.2':
Issue 12086: add example showing how to use name mangling.
http://hg.python.org/cpython/rev/79f9698777b7

New changeset fca745bc70be by Raymond Hettinger in branch 'default':
Issue 12086: add example showing how to use name mangling.
http://hg.python.org/cpython/rev/fca745bc70be
msg139073 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-06-25 14:35
Added an example of how to use double underscores correctly.  

I agree with Ezio that the rest of the documentation is clear on the subject.
History
Date User Action Args
2022-04-11 14:57:17adminsetgithub: 56295
2011-06-25 14:35:48rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg139073
2011-06-25 14:31:22python-devsetmessages: + msg139072
2011-06-25 14:28:22python-devsetnosy: + python-dev
messages: + msg139071
2011-05-23 14:32:04eric.araujosetnosy: + eric.araujo
2011-05-17 10:33:46ezio.melottisetmessages: + msg136146
2011-05-17 09:16:49sheepsetmessages: + msg136145
2011-05-17 02:06:51rhettingersetmessages: + msg136133
2011-05-17 02:04:17rhettingersetassignee: lukasz.langa -> rhettinger
2011-05-16 10:17:56lukasz.langasetmessages: + msg136077
2011-05-16 09:34:19ezio.melottisetnosy: + rhettinger, ezio.melotti
messages: + msg136076
2011-05-16 08:54:41sheepsetmessages: + msg136075
2011-05-16 08:52:18tim.goldensetnosy: + tim.golden
messages: + msg136074
2011-05-16 08:42:26georg.brandlsetnosy: + georg.brandl
messages: + msg136073
2011-05-16 08:37:38lukasz.langasetassignee: docs@python -> lukasz.langa
stage: needs patch

nosy: + lukasz.langa
versions: + Python 3.1, Python 3.2, Python 3.3
2011-05-16 08:35:43sheepcreate