Here is a (perhaps incomplete) list of documented names absent in __all__ lists of respective modules. Perhaps some of them (but not all) are worth to be added to __all__ lists.

calendar.Calendar
calendar.HTMLCalendar
calendar.TextCalendar
cgi.test
configparser.Error
csv.unix_dialect
doctest.DocFileCase
doctest.DocTestCase
enum.EnumMeta
fileinput.fileno
ftplib.Error
ftplib.error_perm
ftplib.error_reply
gettext.bind_textdomain_codeset
gettext.lgettext
gettext.lngettext
http.client.HTTPMessage
http.cookies.Morsel
http.server.test
logging.shutdown
mailbox.Error
mailbox.ExternalClashError
mailbox.NoSuchMailboxError
mailbox.NotEmptyError
mimetypes.MimeTypes
optparse.check_choice
pickletools.OpcodeInfo
plistlib.InvalidFileException
pydoc.doc
smtpd.SMTPChannel
subprocess.SubprocessError
subprocess.TimeoutExpired
tarfile.CompressionError
tarfile.HeaderError
tarfile.ReadError
tarfile.open
threading.BrokenBarrierError
tkinter.ttk.Widget
tokenize.open
traceback.FrameSummary
traceback.StackSummary
traceback.TracebackException
traceback.walk_stack
traceback.walk_tb
wave.Wave_read
wave.Wave_write
xml.etree.ElementTree.XMLPullParser

http.client.HTTPMessage: See Issue 23439. There was resistance to adding this (and the status code constants), though IMO they should be added, since they are documented public APIs.

http.server.test(): In Issue 23418, I consciously left this function out. It is only mentioned as a place to look for sample code, as far as I can tell.

New changeset ebf3e6332a44 by Berker Peksag in branch 'default':
Issue #23883: Add missing entries to traceback.__all__.
https://hg.python.org/cpython/rev/ebf3e6332a44

May be makes sense to add a helper in test.support that implements a test similar to the one in issue23411, and add tests for __all__ in multiple modules.

I working on these three. 
calendar.Calendar
calendar.HTMLCalendar
calendar.TextCalendar

Changes would be the same as https://hg.python.org/cpython/rev/ebf3e6332a44/ for every module?

Serhiy: Yes I was also thinking it might be time for a common helper function.

Milap: I think changes like you mentioned (originally by me) would be fine. Another variation was done for Issue 10838: revision 10b0a8076be8, which expects each object that is not a module object (e.g. not from “import sys”), rather than expecting each object that is a function or class defined in the module. It might depend on the particular circumstance which technique is superior.

New changeset 86fbe140e395 by Andrew Kuchling in branch '2.7':
#23883: add names missing from __all__ (l*gettext, bind_textdomain_codeset)
https://hg.python.org/cpython/rev/86fbe140e395

New changeset 717d87c13f0d by Andrew Kuchling in branch '3.4':
#23883: add names missing from __all__ (l*gettext, bind_textdomain_codeset)
https://hg.python.org/cpython/rev/717d87c13f0d

I took care of the tarfile module.

Added the following according to the first message:

tarfile.CompressionError
tarfile.HeaderError
tarfile.ReadError
tarfile.open


The following were included in __all__ that were not explicitly mentioned in the first message but were denoted as an exported function, exported class, or an exported error.

tarfile.main
tarfile.TarIter
tarfile.StreamError
tarfile.ExtractError
tarfile.SubsequentHeaderError
tarfile.InvalidHeaderError
tarfile.EmptyHeaderError
tarfile.EOFHeaderError
tarfile.TruncatedHeaderError

This is my first patch so feedback is highly appreciated.

Regarding tarfile: Two of the extra errors are documented, so I agree they should be added:

* tarfile.StreamError
* tarfile.ExtractError

However I’m not so sure about main(), TarIter, and the HeaderError subclasses. They aren’t mentioned in the documentation. At least main() and TarIter are just implementation details I think.

There are other documented items that should be added in my opinion. These would not be picked up by the proposed test, although they would be picked up by making the test like in revision 10b0a8076be8.

* ENCODING
* USTAR/GNU/PAX/DEFAULT_FORMAT

Thanks for the feedback.  I was unsure how to proceed with the undocumented items that seemed to be categorized as exported.  Thanks for catching ENCODING & *_FORMAT.

Oh I see, TarIter is listed underneath a comment saying “Exported Classes”, and main() is listed underneath “exported functions”. If they are indeed meant to be exported, they should probably also be documented. Otherwise, maybe we can just add another comment clarifying that they are internal.

In the latest patch, I think HeaderError should be added back to __all__; it is just its subclasses that are not documented. Also XHDTYPE is listed twice in the test case.

If it were up to me, I would add the TarInfo.type constants (REGTYPE, AREGTYPE, LNKTYPE, SYMTYPE, DIRTYPE, FIFOTYPE, CONTTYPE, CHRTYPE, BLKTYPE, GNUTYPE_SPARSE). But I’m not sure if others would agree.

Put HeaderError back in and removed the extra XHDTYPE.

We can get more input on the type constants as well as the undocumented but exported items.  Could just be cleared up with some edits to documentation.

I took a stab at the calendar module.  Found a few items in the documentation which weren't listed in the above list:

LocaleTextCalendar, LocaleHTMLCalendar, and weekheader.

I was curious though about week and prweek as month and prmonth are documented and exported should we add week and prweek to the export & docs?

Woops just noticed above in the issue someone else picked up the Calendar __all__.  I am genuinely sorry I didn't intend to duplicate the effort.

Hi guys! 

Here is a patch for the fileinput module, with some names beyond fileinput.fileno: fileinput.hook_compressed, fileinput.hook_encoded as mentioned in the docs https://docs.python.org/3/library/fileinput.html

This is my first patch as well, so feedback's appreciated!

Hi! This is my first attempt at contributing so as always, feedback will be well appreciated. :)

I meant to start small so I took a shot with csv module. In test, initial expected set contains QUOTE_* because they don't provide __module__ attribute, and __doc/version__ because they are already in csv.__all__ (should they?).

I've made a few PEP8-related fixes just around code I've touched (so they aren't completely unrelated). Is that ok?

Reviews of the patches waiting here:

tarfile, calendar (Joel): Look mainly good; I added minor suggestions about the test cases to Reitveld.

fileinput (Mauro): Looks pretty good; one minor comment on Rietveld.

csv (Jacek): Pretty good; couple minor suggestions. In a perfect world, I don’t think __doc/version__ should be there, but since they are already there it is probably safer to leave them. In general, I think style fixes in related code are okay; although in this case I have no problem with the original single blank lines.

There is nothing seriously wrong with the patches so far. They could be committed, perhaps with a few of the tweaks I suggested.

Summary of other things mentioned here left to do:

* Common test.support helper function
* gettext: Module fixed, but no changes to test suite
* Remaining modules from Serhiy’s list: cgi, configparser, doctest, enum, ftplib, http.cookies, logging, mailbox, mimetypes, optparse, pickletools, plistlib, pydoc, smtpd, subprocess, threading, tkinter.ttk, tokenize, wave, xml.etree.ElementTree.

One outstanding question is what to do about module-level constants that are only briefly mentioned in the documentation. IMO they should be included in __all__. Examples: http.client statuses <https://docs.python.org/3.4/library/http.client.html#http.client.HTTPS_PORT>, tarfile member types <https://docs.python.org/dev/library/tarfile.html#tarfile.TarInfo.type>, calendar weekdays <https://docs.python.org/dev/library/calendar.html#calendar.setfirstweekday>. Precedent of similar constants that are already included, in native Python modules: io SEEK_ constants; lzma FORMAT_, CHECK_, PRESET_ etc.

Thank you for feedback, Martin. I've amended the the patch.

Next, I've prepared some initial test.support.check__all__ helper, based on generalization of all previous patches. Its name/params' descriptions may be a bit rough - amendments/suggestions for such will be strongly appreciated: Issue23883_support_check__all__.patch

I've added missing test.test_gettext.MiscTestCase, based on aforementioned check__all__ helper: Issue23883_test_gettext.patch 

I've also took the liberty of working on some more modules. These are: csv (using new helper), enum, ftplib, logging, optparse, pickletools, threading and wave: Issue23883_all.patch

ftplib and threading have more functions (missing in their __all__ variables) that appear to be documented than mentioned in msg240217 - namely:
* ftplib.error_temp https://docs.python.org/3/library/ftplib.html#ftplib.error_temp
* ftplib.error_proto https://docs.python.org/3/library/ftplib.html#ftplib.error_proto
* threading.main_thread https://docs.python.org/3/library/threading.html#threading.main_thread

so I've added them as well.

> ftplib and threading have more functions

I've meant function and exceptions, of course. Sorry for the noise.

I've added previously missing test and docs for test.support.check__all__ in Issue23883_support_check__all__.v2.patch . Awaiting review. :)

To avoid the list of patches here getting out of control, I suggest opening a fresh issue for any new patches that aren’t a new version of the patches here. We can mark the new issue as a dependency of this one to keep track of it.

Nice work with the check__all__() function. I left some comments on Reitveld. Also, it currently ignores items satisfying either of these checks:

* isinstance(module_object, types.ModuleType)
* getattr(module_object, '__module__', None) not in name_of_module

The first is largely redundant with the second, because module objects don’t have a __module__ attribute. However I wonder if it would be better to drop the second check and just rely on the ModuleType check, making the test stricter. Or would this be too annoying in some cases (requiring a huge blacklist)? If so, maybe make the name_of_module checking optional.

===

Serhiy: ftplib.Error does not actually appear to be documented. Perhaps it should not be added to __all__ after all? (excuse the pun)

> Nice work with the check__all__() function.

Thank you! :)

> I left some comments on Reitveld. Also, it currently ignores items satisfying either of these checks:
>
> * isinstance(module_object, types.ModuleType)
> * getattr(module_object, '__module__', None) not in name_of_module
>
> The first is largely redundant with the second, because module objects don’t have a __module__ attribute. However I wonder if it would be better to drop the second check and just rely on the ModuleType check, making the test stricter. Or would this be too annoying in some cases (requiring a huge blacklist)? If so, maybe make the name_of_module checking optional.

Could you please elaborate on "making the test stricter"?

I'd go with the first check + optional name_of_module. With second one alone, all freshly added test__all__ tests would need additional names in blacklists - not huge ones, but they would otherwise be unnecessary.
  
I've amended the patches and I'm waiting for review.

I've also thought of not only making name_of_module param optional, but to make it extra_names_of_module (so such param would be added to module.__name__ used in "getattr(module_object, '__module__', None) in name of module" check. It would account for less typing in general (module.__name__ occurs in almost all cases), but also less explicity. What do you think?

> Serhiy: ftplib.Error does not actually appear to be documented. Perhaps it should not be added to __all__ after all? (excuse the pun)

Agree. The list is only cursorily filtered result of some one-liners and can contain false names.

Adding new names to __all__ can have undesired effect and break user code (by hiding builtins as for tarfile.open). Perhaps not all documented names should be imported with "import *". In any case it is too late for 3.5.

I think names should be in __all__ even if they shadow builtins, at least in a new feature release. There is plenty of precedent, e.g. asyncio.TimeoutError; reprlib.repr(); threading.enumerate(). Modules with open() in __all__ include aifc, bz2, codecs, dbm, dbm.dumb, gzip, lzma, os, shelve, wave and webbrowser. Plus, pydoc ignores things excluded from __all__.

Jacek: If we used the ModuleType check, and somebody adds a module-level constant (like logging.CRITICAL = 50), the test will automatically detect if they forget to update __all__. That is what I meant by the test being stricter. But it looks like you went for the other option, which has its own relative advantages :)

Will try to to a proper review later when I get a chance.

The technical bit of Issue23883_support_check__all__.v3.patch looks pretty good. Mainly some grammar suggestions for the documentation.

Issue23883_test_gettext.v2.patch looks fine; just depends on check__all__() being added.

Couple of comments about the APIs for ftplib and threading. The changes for the other modules all look good though.

Regarding name_of_module, no strong opinion, but maybe keep it as it is for simplicity. You only used it once so far I think anyway.

> In any case it is too late for 3.5.

Ok, next round of patches is based on default branch.

> Jacek: If we used the ModuleType check, and somebody adds a module-level constant (like logging.CRITICAL = 50), the test will automatically detect if they forget to update __all__. That is what I meant by the test being stricter.

Right and I think such case should be covered as well. I think it may be worth the hassle of adding new condition in detecting names expected to be documented, so the whole if clause would look like:

if (getattr(module_object, '__module__', None) in name_of_module
        or (not isinstance(module_object, types.ModuleType)
            and not hasattr(module_object, '__module__'))):
    expected.add(name)

Obviously tradeoff lies in required blacklisting:
* with previous __module__ check - all undocumented, non "_*" names defined in checked module, but constants need to be in *extra* and new ones won't be detected
* with ModuleType check only - all undocumented, non "_*" names defined in checked module + all functions and classes imported from other modules needs blacklisting
* with extended __module__ check (proposed above) - all undocumented, non "_*" names defined in checked module + all constants imported from other modules; this choice also requires less 'extra' params (in fact, in these patches only csv.__doc/version__ case left)

In this round of patches I went the new, third way.

One odd thing: in test.test_logging, are these:

3783:        self.addCleanup(setattr, logging, 'raiseExecptions', old_raise)
3790:        self.addCleanup(setattr, logging, 'raiseExecptions', old_raise)

("Ex*ec*ptions") really typos or is it intentional? test.test_logging has raiseExceptions name as well.

Also, pickletools.OpcodeInfo and threading.ThreadError are not really documented, are they?

That raiseExecptions thing looks like a typo to me. The code should probably be monkey patching the module variable, and restoring it after the test. Then you wouldn’t need to add your extra typoed version to the blacklist.

In the logging module, I reckon raiseExceptions (non-typoed) should actually be added to __all__. It is documented under Handler.handleError().

pickletools.OpcodeInfo: It is briefly mentioned as the type of the first item of genops(). I don’t have a strong opinion, but I tended to agree with your previous patch which added it to __all__.

threading.ThreadError: It is not documented, but it was already in __all__. I think it should be restored, in case someone’s code is relying on it.

I'm getting patches ready with amendments you've proposed. Two things, though (and two on Rietveld):

> That raiseExecptions thing looks like a typo to me. The code should probably be monkey patching the module variable, and restoring it after the test. Then you wouldn’t need to add your extra typoed version to the blacklist.

Wouldn't it be better to just blacklist the typoed version in this patch, with proper comment, and then fix the typo along with test? Working it around like you proposed looks like unnecessary overkill. I'm also not yet sure where is the "don't change too much in one patch" border.

> pickletools.OpcodeInfo: It is briefly mentioned as the type of the first item of genops(). I don’t have a strong opinion, but I tended to agree with your previous patch which added it to __all__.

That addition was a little absentminded of me, sorry for that. Is such brief mention considered a documentation for a part of API in this case?

raiseExecptions typo: Might be best to get the typo fixed first (maybe open a separate issue, since it should probably be fixed starting from the 3.4 branch).

Regarding OpcodeInfo, it is probably up to your judgement.

> raiseExecptions typo: Might be best to get the typo fixed first (maybe open a separate issue, since it should probably be fixed starting from the 3.4 branch).

Done in #24678 and commited in 83b45ea19d00 .

> Regarding OpcodeInfo, it is probably up to your judgement.

Then I'll leave it as it was - without OpcodeInfo in pickletools.__all__ . The test for it remains in the patch, though.

Thank you all for your work and apologies for my lack of response.

I'm +1 on adding a check__all__ helper to test.support. But passing "self" to it feels a bit weird. Perhaps the assertCountEqual part could be moved outside of the helper. If Serhiy(and/or other people) are happy with the current API, I am happy too :)

Here is a brainstorm of alternatives that don’t require passing “self” into a helper function. But IMO the current proposal that does pass “self” is better.

* Passive expected_module_api() function, and manually check the return value. Precedent: support.detect_api_mismatch().

def test_all(self):  # In class test.test_tarfile.MiscTest
    blacklist = {"bltn_open", ...}
    possible_exports = support.expected_module_api(tarfile, ignore=blacklist)
    self.assertCountEqual(ftplib.__all__, possible_exports)

* ModuleApiTestBase class. Subclass it to use it:

class ExportsTest(support.ModuleApiTestBase):  # In module test.test_tarfile
    module = tarfile
    ignore = {"bltn_open", ...}

* Raise AssertionError directly in case of failure. No automatic error message showing the different names though. Precedents: support.run_doctest(), .check_warnings(), script_helper.assert_python_ok(), _failure().

* Make a temporary internal TestCase instance:

def check__all__(module, etc):
    expected = ...
    ...
    TestCase().assertCountEqual(module.__all__, expected)

Does anyone have strong preference towards one of the propositions above?

TestCase subclass looks reasonable IMHO, but I'd not add that to the scope of this issue (I'd be happy to implement it later, though).

Any suggestions?

many things are not present in os.__all__ that should be, including
os.getcwd

Michael: According to Issue 18554, os.__all__ was fixed in 3.5. Can you confirm? It is working for me:

Python 3.5.0 (default, Sep 20 2015, 11:28:25) 
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> "getcwd" in os.__all__
True

@Martin, my mistake. You're correct. I forgot I was using Python v3.4.

Berker (or anyone else), do you have a preference on how we move forward? I am inclined to use Jacek’s function as it is. I think it is certainly an improvement over the current state. People can propose an alternative version of the function later if they want, though in my opinion the underlying problem is in the architecture of unittest’s assertion methods; see Issue 19645.

I have added comments on Rietveld. Besides few stylistic nitpicks Issue23883_support_check__all__.v5.patch LGTM.

> But passing "self" to it feels a bit weird.

This is not new. There are other testing helpers in test.support that needs passing "self". If the helper is used many times in one test class, I prefer to make a method:

    class SomeTest(TestCase):
        check_something = test.support.check_something

        def test_foo():
            self.check_something('foo')

        def test_bar():
            self.check_something('bar')

But in this case I'm happy with the current API.

Serhiy, thank you for the review. I've made proposed changes (along with rebasing Issue23883_all patch; Issue23883_test_gettext.v3.patch still applies cleanly).

I like Martin's support.expected_module_api() suggestion in msg247167. I still find passing self to a function just to use assertCountEqual a bit weird, but I can live with that.

The reason why I prefer the current API over my support.expected_module_api() idea is it requires the extra assertCountEqual() boilerplate at each call site.

Jacek’s three patches look ready to me. I propose:

1. Commit Issue23883_support_check__all__.v6.patch to 3.6, which everything else depends on.

2. Commit Issue23883_test_gettext.v3.patch to 3.6. (Andrew Kuchling’s original gettext.__all__ fix was made in 3.4 and 2.7 as well, but we would have to backport the support function, or rewrite the test, to apply this to earlier branches.)

3. Commit Issue23883_all.v6.patch to 3.6 only to limit the chance of breaking existing code.

4. Rewrite Mauro SM Rodrigues’s issue23883_fileinput.patch to use support.check__all__().

5. Update Joel Taddei’s Issue23883_tarfile_all.patch and Issue23883_calendar_all.patch for support.check__all__() and addressing review comments.

6. Work on the remaining modules, probably in a separate issue to keep things under control. According to my calculations these modules are: cgi, configparser, doctest, http.cookies, mailbox, mimetypes, plistlib, pydoc, smtpd, tkinter.ttk, tokenize, xml.etree.ElementTree.

Another question that comes to mind: Should we add anything into What’s New, maybe warning of new symbols from “import *”?

> The reason why I prefer the current API over my support.expected_module_api() idea is it requires the extra assertCountEqual() boilerplate at each call site.

I personally find explicit assert* calls in a test case more readable(e.g. I don't need to check what the helper does every N month), but I guess that's another version of tabs vs. space debate :)

> Jacek’s three patches look ready to me. I propose:

Your plan sounds good to me. Thanks!

> Should we add anything into What’s New, maybe warning of new symbols from “import *”?

I guess it wouldn't hurt to add a sentence :)

New changeset f8fa7bc837a3 by Martin Panter in branch 'default':
Issue #23883: Add test.support.check__all__() and test gettext.__all__
https://hg.python.org/cpython/rev/f8fa7bc837a3

New changeset 78d67bdc1142 by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to __all__; patch by Jacek Kołodziej
https://hg.python.org/cpython/rev/78d67bdc1142

New changeset 25a7ceed79d1 by Martin Panter in branch 'default':
Issue #23883: Add news listing modules with new exported APIs
https://hg.python.org/cpython/rev/25a7ceed79d1

Thankyou for sticking with this Jacek. I have committed your three patches. I reworded the documentation a little bit, mainly so it says that it looks for “public” names, rather than documented names, because it does not look at documentation at all. I also added a note to the Changes in the Python API section of What’s New.

Next step I think is to finish off those patches for tarfile, calendar and fileinput.

Martin, yay! :) And thank you for the documentation correction.

Milap, Joel, Mauro, are you still interested in working on patches for calendar/tarfile/fileinput patches? I intend to finish them up if that's not the case.

Yes, I'm, I have a commitment now but I'll submit a new version later today

New version.

issue23883_fileinput.v2.patch looks good to me.

Week and no response, I'm posting updated patches for calendar and tarfile.

I committed the last three patches to 3.6:

571632315c36: fileinput
a2ffa9eedb1b: calendar
48090e08e367: tarfile
a5d3ebb6ad2a: Update news

Please let me know if there are some outstanding patches here that I missed. Otherwise, I think we are up to step 6 in <https://bugs.python.org/issue23883#msg254581>.

New changeset 571632315c36 by Martin Panter in branch 'default':
Issue #23883: Missing fileinput.__all__ APIs; patch by Mauro SM Rodrigues
https://hg.python.org/cpython/rev/571632315c36

New changeset a2ffa9eedb1b by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to calendar.__all__
https://hg.python.org/cpython/rev/a2ffa9eedb1b

New changeset 48090e08e367 by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to tarfile.__all__
https://hg.python.org/cpython/rev/48090e08e367

New changeset a5d3ebb6ad2a by Martin Panter in branch 'default':
Issue #23883: Update news
https://hg.python.org/cpython/rev/a5d3ebb6ad2a

New changeset 62e925be0aff by Serhiy Storchaka in branch 'default':
Issue #23883: Removed redundant names from blacklists.
https://hg.python.org/cpython/rev/62e925be0aff

Thanks for caring for this Martin.

> Should we add anything into What's New, maybe warning of new symbols from "import *"?

I think yes.

New changeset bd6127a6354f by Martin Panter in branch 'default':
Issue #23883: grp and pwd are None on Windows
https://hg.python.org/cpython/rev/bd6127a6354f

Serhiy: I already added a bullet point at <https://docs.python.org/3.6/whatsnew/3.6.html#changes-in-the-python-api>.

Per Martin's request, I've created a few new issues for next batch of module's __all__ list updates:
* cgi: #27105
* configparser: #27106
* mailbox: #27107
* mimetypes: #27108
* plistlib: #27109
* smtpd: #27110
* tokenize: #27112

I've also looked at pydoc module, but I'm not sure what to do with it: `doc` function has only a brief docstring, it's not mentioned in docs at all. Should it really be in pydoc.__all__?

I think pydoc could be left alone. The RST documentation doesn’t say anything about importing any functions from the module that I can see. I was surprised that it even defines __all__ = ["help"]. Perhaps pydoc.doc() was another false indication in Serhiy’s list.

In this case I'm proposing a small patch just for testing pydoc module's __all__ list and left the decision to you, whether to apply it or not. :)

Test doesn't use test.support.check__all__ (see msg266312) - blacklist would be huge and expected list, as you already pointed out, has only one value.

New changeset a36c7f87eba9 by Martin Panter in branch 'default':
Issue #23883: News updates for __all__ attributes
https://hg.python.org/cpython/rev/a36c7f87eba9

Can this issue be closed now?