classification
Title: Enhance dir(module) to be informed by __all__ by updating module.__dir__
Type: enhancement Stage: resolved
Components: Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Implement PEP 562: module __getattr__ and __dir__
View: 32225
Assigned To: Nosy List: brett.cannon, cheryl.sabella, codypiersall, ncoghlan, r.david.murray, rhettinger, serhiy.storchaka, terry.reedy
Priority: normal Keywords:

Created on 2017-09-18 02:20 by codypiersall, last changed 2018-12-21 23:21 by cheryl.sabella. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3610 closed codypiersall, 2017-09-18 02:20
Messages (8)
msg302404 - (view) Author: Cody Piersall (codypiersall) * Date: 2017-09-18 02:20
If a some_module defines __all__, dir(some_module) should only return what is in __all__.  This is already a mechanism that Python provides to specify module-level APIs.  Currently, dir(some_module) returns some_module.__dict__.keys().

The concern with this enhancement is backwards compatibility.  It is conceivable that some library's code would be broken with the different return value of dir(some_module).  However, it seems unlikely that any code, other than tests, depends on the current behavior of dir(some_module).

If __all__ is not defined in some_module, the old behavior is preserved.
msg302405 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-09-18 02:36
I think  change this would be risky and could break code that uses dir() to actually know what is in a module.

IIRC, this was discussed when __all__ was introduced. I believe it was decided that __all__ would affect from-imports and help() but not dir().  This decision has been in place for a long time now and I would expect that people (and tools) have come to rely on the current behavior.
msg302406 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-09-18 03:37
Guido on python-ideas: I ave to agree with the other committers who already spoke up.

Me: I use dir to see the complete list now presently presented.  The presence or absence of  __file__ indicates coded on Python or not.

IDLE module completion start with public items (I believe __all__ or not start with _).  Type '_' and the complete list appears.  I think we should reject this.
msg302407 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-09-18 03:42
While I agree changing the default would be risky from a compatibility perspective, I do think it would be helpful to offer a straightforward way to filter out transitive imports and other non-public implementation details from the result of `dir(module)`, and require folks to use `list(vars(module).keys())` if they want a reliably complete listing of all global variables defined in a module.

The simplest spelling that comes to mind would be to allow people to write:

    __dir__ = __all__

to get ``__all__`` to also affect the result of ``dir()``, permitting module authors to make their own determination as to whether they consider "The result of dir() includes non-public implementation details" to be covered by any backwards compatibility guarantees they might offer.

A less simple, but potentially more flexible, mechanism would be to require that to be spelled as:

    def __dir__():
        return __all__

but I'm not sure the extra flexibility would be useful in any meaningful way (especially if one of the options for easier definition of dynamic module level properties were eventually adopted).
msg302408 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-09-18 05:02
As far the implementation goes, since the `__dir__` overload support in the `dir()` builtin ignores instance dictionaries, this could be implemented in module.__dir__ (or object.__dir__) rather than directly in the builtin.

Alternatively, it could use a different attribute name rather than having `__dir__` be a list of strings on instances, and a protocol method on classes.

If we went with a different name, then I think `__public__` would be a reasonable option, since the purpose of the attribute is to make it easy to programmatically distinguish public attributes from non-public ones (independently of the established leading underscore convention, which doesn't work well for transitive module references).
msg302409 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-18 06:52
This would break tab completion of names not included in __all__.
msg302425 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-09-18 08:32
Yes, that's one of the goals of the feature: to allow module authors to decide if they want tab completion for all attributes, or just the public API.
msg332309 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-12-21 23:21
The OP commented on the PR that a feature close enough to the original request was being implemented in PEP562, so I will close this issue with that one as a superseder.
History
Date User Action Args
2018-12-21 23:21:07cheryl.sabellasetstatus: open -> closed

superseder: Implement PEP 562: module __getattr__ and __dir__

nosy: + cheryl.sabella
messages: + msg332309
resolution: duplicate
stage: resolved
2017-09-18 13:55:33r.david.murraysetnosy: + r.david.murray
2017-09-18 08:32:39ncoghlansetmessages: + msg302425
2017-09-18 06:52:06serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg302409
2017-09-18 05:02:29ncoghlansetmessages: + msg302408
2017-09-18 03:42:20ncoghlansetnosy: + ncoghlan
messages: + msg302407
2017-09-18 03:37:32terry.reedysetnosy: + terry.reedy
messages: + msg302406
2017-09-18 02:36:14rhettingersetnosy: + rhettinger, brett.cannon
messages: + msg302405
2017-09-18 02:20:36codypiersallcreate