classification
Title: Documentation incorrectly suggests __init__ called after direct __new__ call
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Aaron.Staley, Ankur.Ankan, chris.jerdonek, docs@python, nanjekyejoannah, rhettinger
Priority: normal Keywords: patch

Created on 2012-08-02 22:23 by Aaron.Staley, last changed 2019-08-26 07:20 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15478 merged nanjekyejoannah, 2019-08-24 23:19
PR 15506 merged miss-islington, 2019-08-26 06:53
Messages (7)
msg167267 - (view) Author: Aaron Staley (Aaron.Staley) Date: 2012-08-02 22:23
The documentation for __new__ at http://docs.python.org/reference/datamodel.html#object.__new__ is:
"""
object.__new__(cls[, ...])
Called to create a new instance of class cls. __new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument. The remaining arguments are those passed to the object constructor expression (the call to the class). The return value of __new__() should be the new object instance (usually an instance of cls).

Typical implementations create a new instance of the class by invoking the superclass’s __new__() method using super(currentclass, cls).__new__(cls[, ...]) with appropriate arguments and then modifying the newly-created instance as necessary before returning it.

If __new__() returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__().

If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked.

__new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.
"""


The problem is in this line: "If __new__() returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__()."

This is only true in the context of a constructor. In particular, directly calling cls.__new__(cls) will NOT call __init__.

If I define a class:
  
class C(object):
  def __new__(*args, **kwargs):
    print 'new', args, kwargs
    return object.__new__(*args,**kwargs)
  def __init__(self):
    print 'init'

C() will result in __new__ and __init__ being both executed, but C.__new__(C) will only create the instance of C; it will not call __init__!

The original documentation described in http://bugs.python.org/issue1123716 was more correct:

"__new__ must return an object... If you return an
existing object, the constructor call will still call
its __init__ method unless the object is an instance of
a different class..."

That is __init__ is only being called in the context of an external constructor call.  

Proposed phrasing:
"If __new__() is invoked during object construction (cls()) and it returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to the object constructor."
msg167270 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-08-02 22:59
Whether or not the current language is technically correct, I would support improving its clarity.

Would you like to create a formal patch?  Also, note that the newest documentation is published here:

http://docs.python.org/dev/reference/datamodel.html#object.__new__

(Note that in the future, you do not need to post such full excerpts, just the key parts.)
msg350410 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2019-08-24 23:22
Since this has taken long on the tracker, I just opened a PR with proposed changes from Aaron.
msg350424 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-25 04:42
FWIW, the underlying code for this in Objects/typeobject.c::type_new().
msg350493 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-26 06:53
New changeset 6b16d938d6d1ccb443815e20e8812deed274dc09 by Raymond Hettinger (Joannah Nanjekye) in branch 'master':
bpo-15542: Documentation incorrectly suggests __init__ called after direct  __new__ call (GH-15478)
https://github.com/python/cpython/commit/6b16d938d6d1ccb443815e20e8812deed274dc09
msg350502 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-26 07:19
New changeset c841fb9e065c393bba5d5505238f7e286f1dcfc6 by Raymond Hettinger (Miss Islington (bot)) in branch '3.8':
bpo-15542: Documentation incorrectly suggests __init__ called after direct  __new__ call (GH-15478) (GH-15506)
https://github.com/python/cpython/commit/c841fb9e065c393bba5d5505238f7e286f1dcfc6
msg350503 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-26 07:20
Thanks everyone.
History
Date User Action Args
2019-08-26 07:20:23rhettingersetstatus: open -> closed
resolution: fixed
messages: + msg350503

stage: patch review -> resolved
2019-08-26 07:19:23rhettingersetmessages: + msg350502
2019-08-26 06:53:23miss-islingtonsetpull_requests: + pull_request15192
2019-08-26 06:53:15rhettingersetmessages: + msg350493
2019-08-25 04:42:15rhettingersetassignee: docs@python -> rhettinger

messages: + msg350424
nosy: + rhettinger
2019-08-24 23:22:15nanjekyejoannahsetnosy: + nanjekyejoannah
messages: + msg350410
2019-08-24 23:19:32nanjekyejoannahsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request15164
2013-03-03 06:24:49Ankur.Ankansetnosy: + Ankur.Ankan
2012-08-02 22:59:19chris.jerdoneksetversions: + Python 3.3
nosy: + chris.jerdonek

messages: + msg167270

type: enhancement
stage: needs patch
2012-08-02 22:23:43Aaron.Staleycreate