classification
Title: some class attributes missing from dir(Class)
Type: behavior Stage: needs patch
Components: Interpreter Core Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, amaury.forgeotdarc, benjamin.peterson, docs@python, eryksun, georg.brandl, pitrou, siddharthchabra, techdragon
Priority: normal Keywords: patch

Created on 2014-11-04 08:44 by techdragon, last changed 2021-03-27 20:58 by siddharthchabra.

Files
File name Uploaded Description Edit
type_dir_patch.diff georg.brandl, 2014-11-04 10:36 review
Messages (22)
msg230591 - (view) Author: Sam Bishop (techdragon) Date: 2014-11-04 08:44
The output of performing "dir(__class__)" during a class' __init__ method, seems to be lacking the new '__qualname__' attribute in python 3.

This rough test can be pasted right into the python 3.4 REPL to see the issue.
Tested on 64bit python 3.4 running on OSX 10.9
 
"
class Foo:
    class Bar(object):
        keywords = dict()
        def __init__(self, **kwargs):
            print(dir(__class__))
            print(''.join(('"str(__class__.__qualname__)" = ', str(__class__.__qualname__))))
            print(''.join((
                'Is "__qualname__" in the output of the dir() function? : ',
                str(('__qualname__' in dir(__class__))))))
            self.keywords = kwargs


test = Foo.Bar(see='We are missing something here')


"
msg230593 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2014-11-04 09:23
Why did you specify "during class initialization" only? When I print dir(Foo.Bar) at top-level, there is no __qualname__.

Then, note that '__name__' is not listed either, so it's not about new attributes.

It was chosen that dir(someClass) tries to list the know attributes of instances, not the attributes of the class object itself (__bases__, __mro__, and so on).
See https://docs.python.org/3.4/library/functions.html#dir
msg230594 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 09:27
@Amaury: this is not what I read there:

"If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases."

This implies that class attributes are definitely supposed to be in there.
msg230595 - (view) Author: Sam Bishop (techdragon) Date: 2014-11-04 09:29
I specified 'during class initialisation' because that was the only case I confirmed.
msg230596 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 09:29
The missing attributes are some of those defined in type_getsets, i.e.

__name__
__qualname__
__bases__
__abstractmethods__
__text_signature__

The latter two are obscure enough that it probably doesn't matter, but the first three should definitely be there.
msg230597 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 09:31
Ah yes, and some type_members are also missing.
msg230598 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-11-04 09:34
You won't find the __qualname__ data descriptor in dir(Foo.Bar) because it's defined by the metaclass, `type`. Attributes from the metaclass have always been excluded from the dir() of a class.
msg230599 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-11-04 09:38
See type_dir:

https://hg.python.org/cpython/file/ab2c023a9432/Objects/typeobject.c#l2984
msg230602 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 09:59
"Attributes from the metaclass have always been excluded from the dir() of a class."

Be that as it may, I think it is wrong.  I can understand excluding methods of the metaclass, but __qualname__ (and friends) are only defined in the metaclass because they are properties and not __dict__ members, but they are regular attributes of the class, not of the metaclass.

E.g. why is __module__ in there and not __qualname__? Both are determined dynamically at class creation time.

The answer is that it's an implementation detail: there is no tp_module (or ht_module) member in the PyHeapTypeObject struct to store it, so it's stored in __dict__, while __qualname__ is stored in a struct member.
msg230603 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 10:15
BTW, the same implementation detail means that you can ask an instance for its class' __module__, but not the __name__.
msg230605 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 10:23
However, it may not be possible to change this for backward compatibility reasons.  People shouldn't be using dir() for determining attributes and the like, but they do, as documented by the multiprocessing module in the stdlib.

This should at least be noted more explicitly in the docs for dir().
msg230608 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-04 10:28
I definitely think this should be changed. I just don't know how to do it :-)
msg230609 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 10:36
Attaching prototype patch without test suite adjustments.
msg230612 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-04 10:44
I have a question: why would do it for classes and not for regular objects?
msg230613 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-04 10:48
Ah, I misunderstood the patch, sorry. Nevermind.
msg230618 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-11-04 12:30
__doc__ and __module__ are also getsets (to support built-in types), but it's nothing to worry about since the attributes can't be deleted.

I think the most value added here is for listing __mro__ and the others that Georg mentioned. Should the following attributes be blacklisted from dir() as CPython implementation details?

    __base__
    __flags__
    __basicsize__
    __itemsize__
    __dictoffset__
    __weakrefoffset__

It's not as if people will miss what they never had.
msg230639 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2014-11-04 16:52
__base__ exists also in Jython and PyPy (#22456).

I think that all attributes could be listed.
msg230646 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2014-11-04 17:43
So, dir(C) contains '__mro__', but not 'mro'?

I'm -1 on the change.
From https://docs.python.org/3.4/library/functions.html#dir :
"""
Note Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class.
"""

dir(sys) does not list its __str__ method, even if sys.__str__() works, because returning only the explicit content of the module is more interesting to the user.

Likewise, the implementation of dir(__class__) returns the methods and attributes of *instances* because [someone decided that] it's the most relevant info for the user.
msg230647 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 17:46
> So, dir(C) contains '__mro__', but not 'mro'?

That can be discussed.

But I would argue that at least __name__, __bases__ and __qualname__ are interesting attributes for the user.  Same for methods like __subclasses__().

Otherwise, it's quite ironic to prevent attributes that allow introspection in the first place from being displayed in one of the main features used for introspection.
msg230648 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2014-11-04 17:53
Why are __flags__, __basicsize__, __itemsize__, __dictoffset__, and __weakrefoffset__ interesting?

I agree with Georg about the others.
msg230649 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-04 18:00
> Why are __flags__, __basicsize__, __itemsize__, __dictoffset__, and __weakrefoffset__ interesting?

I haven't said they are, but on the other hand I don't see why consistency is a bad thing.
msg388244 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-03-07 16:52
>> Why are __flags__, __basicsize__, __itemsize__, 
>>__dictoffset__, and __weakrefoffset__ interesting?
>
> I haven't said they are, but on the other hand I don't see 
> why consistency is a bad thing.

IMO, they're not interesting because they're not documented attributes in Python's data model. They don't exist in other implementations, such as PyPy. So I'd prefer to hide them and, as much as possible, discourage developers from thinking it's okay to use them. They may be interesting attributes while developing extension modules that define new types, but anyone working at that level surely doesn't need the help of dir() to know what attributes are available.
History
Date User Action Args
2021-03-27 20:58:48siddharthchabrasetnosy: + siddharthchabra

versions: - Python 3.8, Python 3.9, Python 3.10
2021-03-07 16:52:54eryksunsetmessages: + msg388244
components: + Interpreter Core, - Documentation
versions: + Python 3.8, Python 3.9, Python 3.10, - Python 3.4, Python 3.5
2015-07-21 07:47:20ethan.furmansetnosy: - ethan.furman
2014-11-04 18:00:13georg.brandlsetmessages: + msg230649
2014-11-04 17:53:05ethan.furmansetnosy: + ethan.furman
messages: + msg230648
2014-11-04 17:46:55georg.brandlsetmessages: + msg230647
2014-11-04 17:43:40amaury.forgeotdarcsetmessages: + msg230646
2014-11-04 16:52:33Arfreversetmessages: + msg230639
2014-11-04 12:30:04eryksunsetmessages: + msg230618
2014-11-04 11:02:45georg.brandlsetmessages: - msg230614
2014-11-04 10:48:27georg.brandlsetmessages: + msg230614
2014-11-04 10:48:22pitrousetmessages: + msg230613
2014-11-04 10:44:13pitrousetassignee: docs@python ->
messages: + msg230612
2014-11-04 10:40:59Arfreversetnosy: + Arfrever
2014-11-04 10:36:26georg.brandlsetfiles: + type_dir_patch.diff
keywords: + patch
messages: + msg230609
2014-11-04 10:28:38pitrousetnosy: + benjamin.peterson
2014-11-04 10:28:33pitrousetstage: needs patch
messages: + msg230608
versions: + Python 3.5
2014-11-04 10:23:28georg.brandlsetnosy: + docs@python
messages: + msg230605

assignee: docs@python
components: + Documentation, - Interpreter Core
2014-11-04 10:15:39georg.brandlsetmessages: + msg230603
2014-11-04 09:59:59georg.brandlsetnosy: + pitrou
messages: + msg230602
2014-11-04 09:38:39eryksunsetmessages: + msg230599
2014-11-04 09:34:38eryksunsetnosy: + eryksun
messages: + msg230598
2014-11-04 09:31:10georg.brandlsetmessages: + msg230597
2014-11-04 09:29:37georg.brandlsetmessages: + msg230596
2014-11-04 09:29:02techdragonsetmessages: + msg230595
2014-11-04 09:27:05georg.brandlsetstatus: pending -> open
title: __qualname__ missing from dir(__class__) during class initialisation -> some class attributes missing from dir(Class)
nosy: + georg.brandl

messages: + msg230594

resolution: works for me ->
2014-11-04 09:23:14amaury.forgeotdarcsetstatus: open -> pending

nosy: + amaury.forgeotdarc
messages: + msg230593

resolution: works for me
2014-11-04 08:44:04techdragoncreate