classification
Title: dir(closure) does not find __dir__
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Global PyTypeObjects not initialized with PyType_Ready(...)
View: 16369
Assigned To: Nosy List: Arfrever, amaury.forgeotdarc, bfroehle, chris.jerdonek, christopherthemagnificent, jcea, kushal.das
Priority: normal Keywords: patch

Created on 2012-10-17 19:05 by christopherthemagnificent, last changed 2012-10-31 16:16 by jcea. This issue is now closed.

Files
File name Uploaded Description Edit
init_cell_type.patch bfroehle, 2012-10-29 21:01 review
init_cell_type-2.patch amaury.forgeotdarc, 2012-10-30 23:00 review
Messages (6)
msg173197 - (view) Author: Christopher the Magnificent (christopherthemagnificent) Date: 2012-10-17 19:05
This is really short, you should spot the inconsistency in the result of the same function call fairly easily.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Thing:
...     def method(self):
...             print(__class__)
... 
>>> x = Thing.method.__closure__[0]
>>> dir(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object does not provide __dir__
>>> type(x).__dir__(x)
['__gt__', '__eq__', '__setattr__', '__doc__', '__sizeof__', '__str__', '__init__', '__repr__', 'cell_contents', '__dir__', '__ge__', '__class__', '__new__', '__ne__', '__subclasshook__', '__hash__', '__lt__', '__reduce__', '__le__', '__getattribute__', '__format__', '__reduce_ex__', '__delattr__']
>>> dir(x)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'cell_contents']
>>>
msg173225 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-10-18 02:06
Here is a somewhat simpler way to reproduce (following the same definition of x):

>>> dir(x)
  ...
TypeError: object does not provide __dir__
>>> x.__dir__
<built-in method __dir__ of cell object at 0x10c182950>
>>> dir(x)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'cell_contents']

The dir() documentation says, "If the object has a method named __dir__(), this method will be called and must return the list of attributes."

(from http://docs.python.org/py3k/library/functions.html#dir )
msg174152 - (view) Author: Bradley Froehle (bfroehle) * Date: 2012-10-29 21:01
This rather obscure bug seems to be caused by a failure to properly initialize PyCell_Type.

Running with GDB, we see that _PyType_Lookup(<class 'cell'>,  "__dir__") 
fails in:

    /* Look in tp_dict of types in MRO */
    mro = type->tp_mro;

    /* If mro is NULL, the type is either not yet initialized
       by PyType_Ready(), or already cleared by type_clear().
       Either way the safest thing to do is to return NULL. */
    if (mro == NULL)
        return NULL;

Since:

    (gdb) print PyCell_Type->tp_mro
    $9 = (PyObject *) 0x0

Searching the code base shows that we never call PyType_Ready(&PyCell_Type).

A patch is attached.
msg174153 - (view) Author: Bradley Froehle (bfroehle) * Date: 2012-10-29 21:07
Note that we fail to initialize PyCell_Type in all versions of Python, even if there aren't any visible ramifications in earlier versions.
msg174237 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-10-30 23:00
The same fix, but with a unit test.
Note that object.__dir__ did not exist in previous versions, but I found a similar failure with __format__, which also applies to 2.7.
msg174256 - (view) Author: Bradley Froehle (bfroehle) * Date: 2012-10-31 01:50
Your patch looks good to me, and I can verify that it properly tests for the issue in Python 3.3.

On a related note, I've found that PyType_Ready(...) isn't called for a few other core PyTypeObjects... see #16369.
History
Date User Action Args
2012-10-31 16:16:37jceasetnosy: + jcea
2012-10-31 03:35:08benjamin.petersonsetstatus: open -> closed
superseder: Global PyTypeObjects not initialized with PyType_Ready(...)
resolution: duplicate
2012-10-31 01:50:41bfroehlesetmessages: + msg174256
2012-10-30 23:00:30amaury.forgeotdarcsetstage: patch review
2012-10-30 23:00:21amaury.forgeotdarcsetfiles: + init_cell_type-2.patch
nosy: + amaury.forgeotdarc
messages: + msg174237

2012-10-29 21:07:51bfroehlesetmessages: + msg174153
2012-10-29 21:01:32bfroehlesetfiles: + init_cell_type.patch

nosy: + bfroehle
messages: + msg174152

keywords: + patch
2012-10-18 02:06:39chris.jerdoneksetnosy: + chris.jerdonek
title: dir(closure) claims that a closure has no __dir__, only to work later after manually invoking __dir__ from its type -> dir(closure) does not find __dir__
messages: + msg173225

versions: + Python 3.4
2012-10-17 20:17:53Arfreversetnosy: + Arfrever
2012-10-17 19:14:39kushal.dassetnosy: + kushal.das
2012-10-17 19:05:10christopherthemagnificentcreate