classification
Title: __all__ lists are incomplete
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: 10838 26676 26782 27105 27106 27107 27108 27109 27110 27112 Superseder:
Assigned To: Nosy List: Unit03, berker.peksag, martin.panter, maurosr, milap.py, python-dev, r.david.murray, selik, serhiy.storchaka, taddeimania
Priority: normal Keywords: easy, patch

Created on 2015-04-07 16:23 by serhiy.storchaka, last changed 2016-06-06 02:30 by python-dev.

Files
File name Uploaded Description Edit
Issue23883_tarfile_all.patch taddeimania, 2015-04-16 03:47 Add documented items to tarfile.__all__ review
Issue23883_calendar_all.patch taddeimania, 2015-04-16 14:27 Add documented items to calendar.__all__ review
Issue23883_csv_all.patch Unit03, 2015-06-16 16:11 Add documented items to csv.__all__ review
issue23883_fileinput.patch maurosr, 2015-06-17 13:20 Add documented items to fileinput.__all__ review
Issue23883_csv_all.v2.patch Unit03, 2015-06-17 17:49 Add documented items to csv.__all__ review
Issue23883_support_check__all__.patch Unit03, 2015-06-19 01:01 review
Issue23883_test_gettext.patch Unit03, 2015-06-19 01:02 review
Issue23883_all.patch Unit03, 2015-06-19 01:06 review
Issue23883_support_check__all__.v2.patch Unit03, 2015-06-21 16:05 review
Issue23883_support_check__all__.v3.patch Unit03, 2015-06-24 23:19 review
Issue23883_test_gettext.v2.patch Unit03, 2015-06-24 23:19 review
Issue23883_all.v3.patch Unit03, 2015-06-24 23:22 review
Issue23883_support_check__all__.v4.patch Unit03, 2015-07-05 09:51 test.support.check__all__ helper review
Issue23883_test_gettext.v3.patch Unit03, 2015-07-05 09:52 test.test_gettext update review
Issue23883_all.v4.patch Unit03, 2015-07-05 09:57 Add documented items to __all__ in csv, enum, ftplib, logging, optparse, pickletools, threading and wave review
Issue23883_support_check__all__.v5.patch Unit03, 2015-07-22 09:18 test.support.check__all__ helper review
Issue23883_all.v5.patch Unit03, 2015-07-22 09:23 Add documented items to __all__ in csv, enum, ftplib, logging, optparse, threading and wave review
Issue23883_all.v6.patch Unit03, 2015-11-11 08:20 review
Issue23883_support_check__all__.v6.patch Unit03, 2015-11-11 08:22 review
issue23883_fileinput.v2.patch maurosr, 2015-11-16 03:12 Add documented items to fileinput.__all__ review
Issue23883_calendar_all.v2.patch Unit03, 2015-11-21 21:56 review
Issue23883_tarfile_all.v2.patch Unit03, 2015-11-21 21:56 review
Issue23883_pydoc_all.patch Unit03, 2016-05-25 16:49 review
Messages (62)
msg240217 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-07 16:23
Here is a (perhaps incomplete) list of documented names absent in __all__ lists of respective modules. Perhaps some of them (but not all) are worth to be added to __all__ lists.

calendar.Calendar
calendar.HTMLCalendar
calendar.TextCalendar
cgi.test
configparser.Error
csv.unix_dialect
doctest.DocFileCase
doctest.DocTestCase
enum.EnumMeta
fileinput.fileno
ftplib.Error
ftplib.error_perm
ftplib.error_reply
gettext.bind_textdomain_codeset
gettext.lgettext
gettext.lngettext
http.client.HTTPMessage
http.cookies.Morsel
http.server.test
logging.shutdown
mailbox.Error
mailbox.ExternalClashError
mailbox.NoSuchMailboxError
mailbox.NotEmptyError
mimetypes.MimeTypes
optparse.check_choice
pickletools.OpcodeInfo
plistlib.InvalidFileException
pydoc.doc
smtpd.SMTPChannel
subprocess.SubprocessError
subprocess.TimeoutExpired
tarfile.CompressionError
tarfile.HeaderError
tarfile.ReadError
tarfile.open
threading.BrokenBarrierError
tkinter.ttk.Widget
tokenize.open
traceback.FrameSummary
traceback.StackSummary
traceback.TracebackException
traceback.walk_stack
traceback.walk_tb
wave.Wave_read
wave.Wave_write
xml.etree.ElementTree.XMLPullParser
msg240231 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-07 22:07
http.client.HTTPMessage: See Issue 23439. There was resistance to adding this (and the status code constants), though IMO they should be added, since they are documented public APIs.

http.server.test(): In Issue 23418, I consciously left this function out. It is only mentioned as a place to look for sample code, as far as I can tell.
msg240251 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-08 06:47
New changeset ebf3e6332a44 by Berker Peksag in branch 'default':
Issue #23883: Add missing entries to traceback.__all__.
https://hg.python.org/cpython/rev/ebf3e6332a44
msg240258 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-08 10:51
May be makes sense to add a helper in test.support that implements a test similar to the one in issue23411, and add tests for __all__ in multiple modules.
msg240260 - (view) Author: Milap Bhojak (milap.py) * Date: 2015-04-08 11:01
I working on these three. 
calendar.Calendar
calendar.HTMLCalendar
calendar.TextCalendar

Changes would be the same as https://hg.python.org/cpython/rev/ebf3e6332a44/ for every module?
msg240261 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-08 11:25
Serhiy: Yes I was also thinking it might be time for a common helper function.

Milap: I think changes like you mentioned (originally by me) would be fine. Another variation was done for Issue 10838: revision 10b0a8076be8, which expects each object that is not a module object (e.g. not from “import sys”), rather than expecting each object that is a function or class defined in the module. It might depend on the particular circumstance which technique is superior.
msg240878 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-14 13:10
New changeset 86fbe140e395 by Andrew Kuchling in branch '2.7':
#23883: add names missing from __all__ (l*gettext, bind_textdomain_codeset)
https://hg.python.org/cpython/rev/86fbe140e395
msg240880 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-14 13:22
New changeset 717d87c13f0d by Andrew Kuchling in branch '3.4':
#23883: add names missing from __all__ (l*gettext, bind_textdomain_codeset)
https://hg.python.org/cpython/rev/717d87c13f0d
msg241145 - (view) Author: Joel Taddei (taddeimania) * Date: 2015-04-15 19:07
I took care of the tarfile module.

Added the following according to the first message:

tarfile.CompressionError
tarfile.HeaderError
tarfile.ReadError
tarfile.open


The following were included in __all__ that were not explicitly mentioned in the first message but were denoted as an exported function, exported class, or an exported error.

tarfile.main
tarfile.TarIter
tarfile.StreamError
tarfile.ExtractError
tarfile.SubsequentHeaderError
tarfile.InvalidHeaderError
tarfile.EmptyHeaderError
tarfile.EOFHeaderError
tarfile.TruncatedHeaderError

This is my first patch so feedback is highly appreciated.
msg241176 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-15 23:07
Regarding tarfile: Two of the extra errors are documented, so I agree they should be added:

* tarfile.StreamError
* tarfile.ExtractError

However I’m not so sure about main(), TarIter, and the HeaderError subclasses. They aren’t mentioned in the documentation. At least main() and TarIter are just implementation details I think.

There are other documented items that should be added in my opinion. These would not be picked up by the proposed test, although they would be picked up by making the test like in revision 10b0a8076be8.

* ENCODING
* USTAR/GNU/PAX/DEFAULT_FORMAT
msg241195 - (view) Author: Joel Taddei (taddeimania) * Date: 2015-04-16 03:04
Thanks for the feedback.  I was unsure how to proceed with the undocumented items that seemed to be categorized as exported.  Thanks for catching ENCODING & *_FORMAT.
msg241198 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-16 03:35
Oh I see, TarIter is listed underneath a comment saying “Exported Classes”, and main() is listed underneath “exported functions”. If they are indeed meant to be exported, they should probably also be documented. Otherwise, maybe we can just add another comment clarifying that they are internal.

In the latest patch, I think HeaderError should be added back to __all__; it is just its subclasses that are not documented. Also XHDTYPE is listed twice in the test case.

If it were up to me, I would add the TarInfo.type constants (REGTYPE, AREGTYPE, LNKTYPE, SYMTYPE, DIRTYPE, FIFOTYPE, CONTTYPE, CHRTYPE, BLKTYPE, GNUTYPE_SPARSE). But I’m not sure if others would agree.
msg241199 - (view) Author: Joel Taddei (taddeimania) * Date: 2015-04-16 03:47
Put HeaderError back in and removed the extra XHDTYPE.

We can get more input on the type constants as well as the undocumented but exported items.  Could just be cleared up with some edits to documentation.
msg241219 - (view) Author: Joel Taddei (taddeimania) * Date: 2015-04-16 14:27
I took a stab at the calendar module.  Found a few items in the documentation which weren't listed in the above list:

LocaleTextCalendar, LocaleHTMLCalendar, and weekheader.

I was curious though about week and prweek as month and prmonth are documented and exported should we add week and prweek to the export & docs?
msg241306 - (view) Author: Joel Taddei (taddeimania) * Date: 2015-04-17 03:05
Woops just noticed above in the issue someone else picked up the Calendar __all__.  I am genuinely sorry I didn't intend to duplicate the effort.
msg241602 - (view) Author: Mauro S. M. Rodrigues (maurosr) * Date: 2015-04-20 04:25
Hi guys! 

Here is a patch for the fileinput module, with some names beyond fileinput.fileno: fileinput.hook_compressed, fileinput.hook_encoded as mentioned in the docs https://docs.python.org/3/library/fileinput.html

This is my first patch as well, so feedback's appreciated!
msg245421 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-06-16 13:26
Hi! This is my first attempt at contributing so as always, feedback will be well appreciated. :)

I meant to start small so I took a shot with csv module. In test, initial expected set contains QUOTE_* because they don't provide __module__ attribute, and __doc/version__ because they are already in csv.__all__ (should they?).

I've made a few PEP8-related fixes just around code I've touched (so they aren't completely unrelated). Is that ok?
msg245428 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-06-17 02:42
Reviews of the patches waiting here:

tarfile, calendar (Joel): Look mainly good; I added minor suggestions about the test cases to Reitveld.

fileinput (Mauro): Looks pretty good; one minor comment on Rietveld.

csv (Jacek): Pretty good; couple minor suggestions. In a perfect world, I don’t think __doc/version__ should be there, but since they are already there it is probably safer to leave them. In general, I think style fixes in related code are okay; although in this case I have no problem with the original single blank lines.

There is nothing seriously wrong with the patches so far. They could be committed, perhaps with a few of the tweaks I suggested.

Summary of other things mentioned here left to do:

* Common test.support helper function
* gettext: Module fixed, but no changes to test suite
* Remaining modules from Serhiy’s list: cgi, configparser, doctest, enum, ftplib, http.cookies, logging, mailbox, mimetypes, optparse, pickletools, plistlib, pydoc, smtpd, subprocess, threading, tkinter.ttk, tokenize, wave, xml.etree.ElementTree.

One outstanding question is what to do about module-level constants that are only briefly mentioned in the documentation. IMO they should be included in __all__. Examples: http.client statuses <https://docs.python.org/3.4/library/http.client.html#http.client.HTTPS_PORT>, tarfile member types <https://docs.python.org/dev/library/tarfile.html#tarfile.TarInfo.type>, calendar weekdays <https://docs.python.org/dev/library/calendar.html#calendar.setfirstweekday>. Precedent of similar constants that are already included, in native Python modules: io SEEK_ constants; lzma FORMAT_, CHECK_, PRESET_ etc.
msg245485 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-06-19 01:06
Thank you for feedback, Martin. I've amended the the patch.

Next, I've prepared some initial test.support.check__all__ helper, based on generalization of all previous patches. Its name/params' descriptions may be a bit rough - amendments/suggestions for such will be strongly appreciated: Issue23883_support_check__all__.patch

I've added missing test.test_gettext.MiscTestCase, based on aforementioned check__all__ helper: Issue23883_test_gettext.patch 

I've also took the liberty of working on some more modules. These are: csv (using new helper), enum, ftplib, logging, optparse, pickletools, threading and wave: Issue23883_all.patch

ftplib and threading have more functions (missing in their __all__ variables) that appear to be documented than mentioned in msg240217 - namely:
* ftplib.error_temp https://docs.python.org/3/library/ftplib.html#ftplib.error_temp
* ftplib.error_proto https://docs.python.org/3/library/ftplib.html#ftplib.error_proto
* threading.main_thread https://docs.python.org/3/library/threading.html#threading.main_thread

so I've added them as well.
msg245486 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-06-19 01:10
> ftplib and threading have more functions

I've meant function and exceptions, of course. Sorry for the noise.
msg245598 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-06-21 16:05
I've added previously missing test and docs for test.support.check__all__ in Issue23883_support_check__all__.v2.patch . Awaiting review. :)
msg245614 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-06-22 02:33
To avoid the list of patches here getting out of control, I suggest opening a fresh issue for any new patches that aren’t a new version of the patches here. We can mark the new issue as a dependency of this one to keep track of it.

Nice work with the check__all__() function. I left some comments on Reitveld. Also, it currently ignores items satisfying either of these checks:

* isinstance(module_object, types.ModuleType)
* getattr(module_object, '__module__', None) not in name_of_module

The first is largely redundant with the second, because module objects don’t have a __module__ attribute. However I wonder if it would be better to drop the second check and just rely on the ModuleType check, making the test stricter. Or would this be too annoying in some cases (requiring a huge blacklist)? If so, maybe make the name_of_module checking optional.

===

Serhiy: ftplib.Error does not actually appear to be documented. Perhaps it should not be added to __all__ after all? (excuse the pun)
msg245784 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-06-24 23:22
> Nice work with the check__all__() function.

Thank you! :)

> I left some comments on Reitveld. Also, it currently ignores items satisfying either of these checks:
>
> * isinstance(module_object, types.ModuleType)
> * getattr(module_object, '__module__', None) not in name_of_module
>
> The first is largely redundant with the second, because module objects don’t have a __module__ attribute. However I wonder if it would be better to drop the second check and just rely on the ModuleType check, making the test stricter. Or would this be too annoying in some cases (requiring a huge blacklist)? If so, maybe make the name_of_module checking optional.

Could you please elaborate on "making the test stricter"?

I'd go with the first check + optional name_of_module. With second one alone, all freshly added test__all__ tests would need additional names in blacklists - not huge ones, but they would otherwise be unnecessary.
  
I've amended the patches and I'm waiting for review.

I've also thought of not only making name_of_module param optional, but to make it extra_names_of_module (so such param would be added to module.__name__ used in "getattr(module_object, '__module__', None) in name of module" check. It would account for less typing in general (module.__name__ occurs in almost all cases), but also less explicity. What do you think?
msg245794 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-06-25 04:26
> Serhiy: ftplib.Error does not actually appear to be documented. Perhaps it should not be added to __all__ after all? (excuse the pun)

Agree. The list is only cursorily filtered result of some one-liners and can contain false names.
msg245796 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-06-25 04:39
Adding new names to __all__ can have undesired effect and break user code (by hiding builtins as for tarfile.open). Perhaps not all documented names should be imported with "import *". In any case it is too late for 3.5.
msg245801 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-06-25 05:37
I think names should be in __all__ even if they shadow builtins, at least in a new feature release. There is plenty of precedent, e.g. asyncio.TimeoutError; reprlib.repr(); threading.enumerate(). Modules with open() in __all__ include aifc, bz2, codecs, dbm, dbm.dumb, gzip, lzma, os, shelve, wave and webbrowser. Plus, pydoc ignores things excluded from __all__.
msg245802 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-06-25 06:02
Jacek: If we used the ModuleType check, and somebody adds a module-level constant (like logging.CRITICAL = 50), the test will automatically detect if they forget to update __all__. That is what I meant by the test being stricter. But it looks like you went for the other option, which has its own relative advantages :)

Will try to to a proper review later when I get a chance.
msg246024 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-01 03:58
The technical bit of Issue23883_support_check__all__.v3.patch looks pretty good. Mainly some grammar suggestions for the documentation.

Issue23883_test_gettext.v2.patch looks fine; just depends on check__all__() being added.

Couple of comments about the APIs for ftplib and threading. The changes for the other modules all look good though.

Regarding name_of_module, no strong opinion, but maybe keep it as it is for simplicity. You only used it once so far I think anyway.
msg246299 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-07-05 09:57
> In any case it is too late for 3.5.

Ok, next round of patches is based on default branch.

> Jacek: If we used the ModuleType check, and somebody adds a module-level constant (like logging.CRITICAL = 50), the test will automatically detect if they forget to update __all__. That is what I meant by the test being stricter.

Right and I think such case should be covered as well. I think it may be worth the hassle of adding new condition in detecting names expected to be documented, so the whole if clause would look like:

if (getattr(module_object, '__module__', None) in name_of_module
        or (not isinstance(module_object, types.ModuleType)
            and not hasattr(module_object, '__module__'))):
    expected.add(name)

Obviously tradeoff lies in required blacklisting:
* with previous __module__ check - all undocumented, non "_*" names defined in checked module, but constants need to be in *extra* and new ones won't be detected
* with ModuleType check only - all undocumented, non "_*" names defined in checked module + all functions and classes imported from other modules needs blacklisting
* with extended __module__ check (proposed above) - all undocumented, non "_*" names defined in checked module + all constants imported from other modules; this choice also requires less 'extra' params (in fact, in these patches only csv.__doc/version__ case left)

In this round of patches I went the new, third way.

One odd thing: in test.test_logging, are these:

3783:        self.addCleanup(setattr, logging, 'raiseExecptions', old_raise)
3790:        self.addCleanup(setattr, logging, 'raiseExecptions', old_raise)

("Ex*ec*ptions") really typos or is it intentional? test.test_logging has raiseExceptions name as well.

Also, pickletools.OpcodeInfo and threading.ThreadError are not really documented, are they?
msg246895 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-18 12:10
That raiseExecptions thing looks like a typo to me. The code should probably be monkey patching the module variable, and restoring it after the test. Then you wouldn’t need to add your extra typoed version to the blacklist.

In the logging module, I reckon raiseExceptions (non-typoed) should actually be added to __all__. It is documented under Handler.handleError().

pickletools.OpcodeInfo: It is briefly mentioned as the type of the first item of genops(). I don’t have a strong opinion, but I tended to agree with your previous patch which added it to __all__.

threading.ThreadError: It is not documented, but it was already in __all__. I think it should be restored, in case someone’s code is relying on it.
msg247031 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-07-21 13:09
I'm getting patches ready with amendments you've proposed. Two things, though (and two on Rietveld):

> That raiseExecptions thing looks like a typo to me. The code should probably be monkey patching the module variable, and restoring it after the test. Then you wouldn’t need to add your extra typoed version to the blacklist.

Wouldn't it be better to just blacklist the typoed version in this patch, with proper comment, and then fix the typo along with test? Working it around like you proposed looks like unnecessary overkill. I'm also not yet sure where is the "don't change too much in one patch" border.

> pickletools.OpcodeInfo: It is briefly mentioned as the type of the first item of genops(). I don’t have a strong opinion, but I tended to agree with your previous patch which added it to __all__.

That addition was a little absentminded of me, sorry for that. Is such brief mention considered a documentation for a part of API in this case?
msg247036 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-21 13:56
raiseExecptions typo: Might be best to get the typo fixed first (maybe open a separate issue, since it should probably be fixed starting from the 3.4 branch).

Regarding OpcodeInfo, it is probably up to your judgement.
msg247102 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-07-22 09:23
> raiseExecptions typo: Might be best to get the typo fixed first (maybe open a separate issue, since it should probably be fixed starting from the 3.4 branch).

Done in #24678 and commited in 83b45ea19d00 .

> Regarding OpcodeInfo, it is probably up to your judgement.

Then I'll leave it as it was - without OpcodeInfo in pickletools.__all__ . The test for it remains in the patch, though.
msg247103 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-07-22 09:37
Thank you all for your work and apologies for my lack of response.

I'm +1 on adding a check__all__ helper to test.support. But passing "self" to it feels a bit weird. Perhaps the assertCountEqual part could be moved outside of the helper. If Serhiy(and/or other people) are happy with the current API, I am happy too :)
msg247167 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-23 05:14
Here is a brainstorm of alternatives that don’t require passing “self” into a helper function. But IMO the current proposal that does pass “self” is better.

* Passive expected_module_api() function, and manually check the return value. Precedent: support.detect_api_mismatch().

def test_all(self):  # In class test.test_tarfile.MiscTest
    blacklist = {"bltn_open", ...}
    possible_exports = support.expected_module_api(tarfile, ignore=blacklist)
    self.assertCountEqual(ftplib.__all__, possible_exports)

* ModuleApiTestBase class. Subclass it to use it:

class ExportsTest(support.ModuleApiTestBase):  # In module test.test_tarfile
    module = tarfile
    ignore = {"bltn_open", ...}

* Raise AssertionError directly in case of failure. No automatic error message showing the different names though. Precedents: support.run_doctest(), .check_warnings(), script_helper.assert_python_ok(), _failure().

* Make a temporary internal TestCase instance:

def check__all__(module, etc):
    expected = ...
    ...
    TestCase().assertCountEqual(module.__all__, expected)
msg251104 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-09-19 18:58
Does anyone have strong preference towards one of the propositions above?

TestCase subclass looks reasonable IMHO, but I'd not add that to the scope of this issue (I'd be happy to implement it later, though).

Any suggestions?
msg254062 - (view) Author: Michael Selik (selik) * Date: 2015-11-04 16:18
many things are not present in os.__all__ that should be, including
os.getcwd
msg254080 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-05 03:53
Michael: According to Issue 18554, os.__all__ was fixed in 3.5. Can you confirm? It is working for me:

Python 3.5.0 (default, Sep 20 2015, 11:28:25) 
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> "getcwd" in os.__all__
True
msg254084 - (view) Author: Michael Selik (selik) * Date: 2015-11-05 06:15
@Martin, my mistake. You're correct. I forgot I was using Python v3.4.
msg254426 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-10 01:22
Berker (or anyone else), do you have a preference on how we move forward? I am inclined to use Jacek’s function as it is. I think it is certainly an improvement over the current state. People can propose an alternative version of the function later if they want, though in my opinion the underlying problem is in the architecture of unittest’s assertion methods; see Issue 19645.
msg254434 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-10 07:20
I have added comments on Rietveld. Besides few stylistic nitpicks Issue23883_support_check__all__.v5.patch LGTM.

> But passing "self" to it feels a bit weird.

This is not new. There are other testing helpers in test.support that needs passing "self". If the helper is used many times in one test class, I prefer to make a method:

    class SomeTest(TestCase):
        check_something = test.support.check_something

        def test_foo():
            self.check_something('foo')

        def test_bar():
            self.check_something('bar')

But in this case I'm happy with the current API.
msg254482 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-11-11 08:22
Serhiy, thank you for the review. I've made proposed changes (along with rebasing Issue23883_all patch; Issue23883_test_gettext.v3.patch still applies cleanly).
msg254517 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-11-12 05:45
I like Martin's support.expected_module_api() suggestion in msg247167. I still find passing self to a function just to use assertCountEqual a bit weird, but I can live with that.
msg254581 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-13 02:07
The reason why I prefer the current API over my support.expected_module_api() idea is it requires the extra assertCountEqual() boilerplate at each call site.

Jacek’s three patches look ready to me. I propose:

1. Commit Issue23883_support_check__all__.v6.patch to 3.6, which everything else depends on.

2. Commit Issue23883_test_gettext.v3.patch to 3.6. (Andrew Kuchling’s original gettext.__all__ fix was made in 3.4 and 2.7 as well, but we would have to backport the support function, or rewrite the test, to apply this to earlier branches.)

3. Commit Issue23883_all.v6.patch to 3.6 only to limit the chance of breaking existing code.

4. Rewrite Mauro SM Rodrigues’s issue23883_fileinput.patch to use support.check__all__().

5. Update Joel Taddei’s Issue23883_tarfile_all.patch and Issue23883_calendar_all.patch for support.check__all__() and addressing review comments.

6. Work on the remaining modules, probably in a separate issue to keep things under control. According to my calculations these modules are: cgi, configparser, doctest, http.cookies, mailbox, mimetypes, plistlib, pydoc, smtpd, tkinter.ttk, tokenize, xml.etree.ElementTree.

Another question that comes to mind: Should we add anything into What’s New, maybe warning of new symbols from “import *”?
msg254585 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-11-13 03:10
> The reason why I prefer the current API over my support.expected_module_api() idea is it requires the extra assertCountEqual() boilerplate at each call site.

I personally find explicit assert* calls in a test case more readable(e.g. I don't need to check what the helper does every N month), but I guess that's another version of tabs vs. space debate :)

> Jacek’s three patches look ready to me. I propose:

Your plan sounds good to me. Thanks!

> Should we add anything into What’s New, maybe warning of new symbols from “import *”?

I guess it wouldn't hurt to add a sentence :)
msg254654 - (view) Author: Roundup Robot (python-dev) Date: 2015-11-14 12:54
New changeset f8fa7bc837a3 by Martin Panter in branch 'default':
Issue #23883: Add test.support.check__all__() and test gettext.__all__
https://hg.python.org/cpython/rev/f8fa7bc837a3

New changeset 78d67bdc1142 by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to __all__; patch by Jacek Kołodziej
https://hg.python.org/cpython/rev/78d67bdc1142

New changeset 25a7ceed79d1 by Martin Panter in branch 'default':
Issue #23883: Add news listing modules with new exported APIs
https://hg.python.org/cpython/rev/25a7ceed79d1
msg254655 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-14 13:05
Thankyou for sticking with this Jacek. I have committed your three patches. I reworded the documentation a little bit, mainly so it says that it looks for “public” names, rather than documented names, because it does not look at documentation at all. I also added a note to the Changes in the Python API section of What’s New.

Next step I think is to finish off those patches for tarfile, calendar and fileinput.
msg254658 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-11-14 13:45
Martin, yay! :) And thank you for the documentation correction.

Milap, Joel, Mauro, are you still interested in working on patches for calendar/tarfile/fileinput patches? I intend to finish them up if that's not the case.
msg254666 - (view) Author: Mauro S. M. Rodrigues (maurosr) * Date: 2015-11-14 19:09
Yes, I'm, I have a commitment now but I'll submit a new version later today
msg254706 - (view) Author: Mauro S. M. Rodrigues (maurosr) * Date: 2015-11-16 03:12
New version.
msg254780 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-17 02:54
issue23883_fileinput.v2.patch looks good to me.
msg255071 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2015-11-21 21:56
Week and no response, I'm posting updated patches for calendar and tarfile.
msg258366 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-16 07:32
I committed the last three patches to 3.6:

571632315c36: fileinput
a2ffa9eedb1b: calendar
48090e08e367: tarfile
a5d3ebb6ad2a: Update news

Please let me know if there are some outstanding patches here that I missed. Otherwise, I think we are up to step 6 in <https://bugs.python.org/issue23883#msg254581>.
msg258367 - (view) Author: Roundup Robot (python-dev) Date: 2016-01-16 07:39
New changeset 571632315c36 by Martin Panter in branch 'default':
Issue #23883: Missing fileinput.__all__ APIs; patch by Mauro SM Rodrigues
https://hg.python.org/cpython/rev/571632315c36

New changeset a2ffa9eedb1b by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to calendar.__all__
https://hg.python.org/cpython/rev/a2ffa9eedb1b

New changeset 48090e08e367 by Martin Panter in branch 'default':
Issue #23883: Add missing APIs to tarfile.__all__
https://hg.python.org/cpython/rev/48090e08e367

New changeset a5d3ebb6ad2a by Martin Panter in branch 'default':
Issue #23883: Update news
https://hg.python.org/cpython/rev/a5d3ebb6ad2a
msg258369 - (view) Author: Roundup Robot (python-dev) Date: 2016-01-16 09:05
New changeset 62e925be0aff by Serhiy Storchaka in branch 'default':
Issue #23883: Removed redundant names from blacklists.
https://hg.python.org/cpython/rev/62e925be0aff
msg258372 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-16 09:13
Thanks for caring for this Martin.

> Should we add anything into What's New, maybe warning of new symbols from "import *"?

I think yes.
msg258381 - (view) Author: Roundup Robot (python-dev) Date: 2016-01-16 11:02
New changeset bd6127a6354f by Martin Panter in branch 'default':
Issue #23883: grp and pwd are None on Windows
https://hg.python.org/cpython/rev/bd6127a6354f
msg258382 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-16 11:08
Serhiy: I already added a bullet point at <https://docs.python.org/3.6/whatsnew/3.6.html#changes-in-the-python-api>.
msg266276 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2016-05-24 21:01
Per Martin's request, I've created a few new issues for next batch of module's __all__ list updates:
* cgi: #27105
* configparser: #27106
* mailbox: #27107
* mimetypes: #27108
* plistlib: #27109
* smtpd: #27110
* tokenize: #27112

I've also looked at pydoc module, but I'm not sure what to do with it: `doc` function has only a brief docstring, it's not mentioned in docs at all. Should it really be in pydoc.__all__?
msg266347 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-05-25 13:16
I think pydoc could be left alone. The RST documentation doesn’t say anything about importing any functions from the module that I can see. I was surprised that it even defines __all__ = ["help"]. Perhaps pydoc.doc() was another false indication in Serhiy’s list.
msg266373 - (view) Author: Jacek Kołodziej (Unit03) * Date: 2016-05-25 16:49
In this case I'm proposing a small patch just for testing pydoc module's __all__ list and left the decision to you, whether to apply it or not. :)

Test doesn't use test.support.check__all__ (see msg266312) - blacklist would be huge and expected list, as you already pointed out, has only one value.
msg267509 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-06 02:30
New changeset a36c7f87eba9 by Martin Panter in branch 'default':
Issue #23883: News updates for __all__ attributes
https://hg.python.org/cpython/rev/a36c7f87eba9
History
Date User Action Args
2016-06-06 02:30:54python-devsetmessages: + msg267509
2016-05-25 16:49:30Unit03setfiles: + Issue23883_pydoc_all.patch

messages: + msg266373
2016-05-25 13:16:53martin.pantersetdependencies: + cgi.__all__ is incomplete, configparser.__all__ is incomplete, mailbox.__all__ list is incomplete, mimetypes.__all__ list is incomplete, plistlib.__all__ list is incomplete, smtpd.__all__ list is incomplete, tokenize.__all__ list is incomplete
messages: + msg266347
2016-05-24 21:01:46Unit03setmessages: + msg266276
2016-04-17 00:24:31martin.pantersetdependencies: + subprocess.__all__ incomplete on Windows
2016-03-31 07:18:10martin.pantersetdependencies: + Add missing XMLPullParser to ElementTree.__all__
2016-01-16 11:08:44martin.pantersetmessages: + msg258382
2016-01-16 11:02:51python-devsetmessages: + msg258381
2016-01-16 09:13:13serhiy.storchakasetmessages: + msg258372
2016-01-16 09:05:48python-devsetmessages: + msg258369
2016-01-16 07:39:25python-devsetmessages: + msg258367
2016-01-16 07:33:00martin.pantersetmessages: + msg258366
2015-11-21 21:56:43Unit03setfiles: + Issue23883_tarfile_all.v2.patch

messages: + msg255071
2015-11-21 21:56:20Unit03setfiles: + Issue23883_calendar_all.v2.patch
2015-11-17 02:54:09martin.pantersetmessages: + msg254780
2015-11-16 03:12:44maurosrsetfiles: + issue23883_fileinput.v2.patch

messages: + msg254706
2015-11-14 19:09:39maurosrsetmessages: + msg254666
2015-11-14 13:45:19Unit03setmessages: + msg254658
2015-11-14 13:05:19martin.pantersetmessages: + msg254655
stage: patch review -> needs patch
2015-11-14 12:54:12python-devsetmessages: + msg254654
2015-11-13 03:10:04berker.peksagsetmessages: + msg254585
2015-11-13 02:07:39martin.pantersetmessages: + msg254581
2015-11-12 05:45:03berker.peksagsetmessages: + msg254517
2015-11-11 08:22:30Unit03setfiles: + Issue23883_support_check__all__.v6.patch

messages: + msg254482
2015-11-11 08:20:36Unit03setfiles: + Issue23883_all.v6.patch
2015-11-10 07:20:03serhiy.storchakasetmessages: + msg254434
2015-11-10 01:22:43martin.pantersetmessages: + msg254426
2015-11-05 06:15:13seliksetmessages: + msg254084
2015-11-05 03:53:37martin.pantersetmessages: + msg254080
2015-11-04 16:18:11seliksetnosy: + selik
messages: + msg254062
2015-09-19 18:58:56Unit03setmessages: + msg251104
2015-07-23 05:14:21martin.pantersetmessages: + msg247167
2015-07-22 09:37:26berker.peksagsetmessages: + msg247103
2015-07-22 09:23:05Unit03setfiles: + Issue23883_all.v5.patch

messages: + msg247102
2015-07-22 09:18:17Unit03setfiles: + Issue23883_support_check__all__.v5.patch
2015-07-21 13:56:48martin.pantersetmessages: + msg247036
2015-07-21 13:09:28Unit03setmessages: + msg247031
2015-07-18 12:10:40martin.pantersetmessages: + msg246895
2015-07-05 09:57:23Unit03setfiles: + Issue23883_all.v4.patch

messages: + msg246299
2015-07-05 09:52:38Unit03setfiles: + Issue23883_test_gettext.v3.patch
2015-07-05 09:51:41Unit03setfiles: + Issue23883_support_check__all__.v4.patch
2015-07-01 03:58:39martin.pantersetmessages: + msg246024
stage: needs patch -> patch review
2015-06-25 06:02:58martin.pantersetmessages: + msg245802
2015-06-25 05:37:54martin.pantersetmessages: + msg245801
2015-06-25 04:39:54serhiy.storchakasetmessages: + msg245796
versions: + Python 3.6, - Python 3.5
2015-06-25 04:26:32serhiy.storchakasetmessages: + msg245794
2015-06-24 23:22:35Unit03setfiles: + Issue23883_all.v3.patch

messages: + msg245784
2015-06-24 23:19:47Unit03setfiles: + Issue23883_test_gettext.v2.patch
2015-06-24 23:19:39Unit03setfiles: + Issue23883_support_check__all__.v3.patch
2015-06-22 02:33:51martin.pantersetmessages: + msg245614
2015-06-21 16:05:36Unit03setfiles: + Issue23883_support_check__all__.v2.patch

messages: + msg245598
2015-06-19 01:10:02Unit03setmessages: + msg245486
2015-06-19 01:06:35Unit03setfiles: + Issue23883_all.patch

messages: + msg245485
2015-06-19 01:02:59Unit03setfiles: + Issue23883_test_gettext.patch
2015-06-19 01:01:58Unit03setfiles: + Issue23883_support_check__all__.patch
2015-06-17 17:49:18Unit03setfiles: + Issue23883_csv_all.v2.patch
2015-06-17 13:24:58maurosrsetfiles: - issue23883_fileinput.patch
2015-06-17 13:20:44maurosrsetfiles: + issue23883_fileinput.patch
2015-06-17 02:42:16martin.pantersetmessages: + msg245428
2015-06-16 16:11:34Unit03setfiles: + Issue23883_csv_all.patch
2015-06-16 16:09:17Unit03setfiles: - Issue23883_csv_all.patch
2015-06-16 13:26:28Unit03setfiles: + Issue23883_csv_all.patch
nosy: + Unit03
messages: + msg245421

2015-04-20 04:25:14maurosrsetfiles: + issue23883_fileinput.patch
nosy: + maurosr
messages: + msg241602

2015-04-17 03:05:41taddeimaniasetmessages: + msg241306
2015-04-16 14:27:41taddeimaniasetfiles: + Issue23883_calendar_all.patch

messages: + msg241219
2015-04-16 03:47:31taddeimaniasetfiles: + Issue23883_tarfile_all.patch

messages: + msg241199
2015-04-16 03:40:26taddeimaniasetfiles: - Issue23883_tarfile_all.patch
2015-04-16 03:35:09martin.pantersetmessages: + msg241198
2015-04-16 03:04:02taddeimaniasetfiles: + Issue23883_tarfile_all.patch

messages: + msg241195
2015-04-16 01:55:22taddeimaniasetfiles: - Issue23883_tarfile_all.patch
2015-04-15 23:07:38martin.pantersetmessages: + msg241176
2015-04-15 19:07:33taddeimaniasetfiles: + Issue23883_tarfile_all.patch

nosy: + taddeimania
messages: + msg241145

keywords: + patch
2015-04-14 13:22:45python-devsetmessages: + msg240880
2015-04-14 13:10:51python-devsetmessages: + msg240878
2015-04-08 11:25:39martin.pantersetmessages: + msg240261
2015-04-08 11:01:43milap.pysetnosy: + milap.py
messages: + msg240260
2015-04-08 10:51:01serhiy.storchakasetmessages: + msg240258
2015-04-08 09:34:38berker.peksagsetkeywords: + easy
stage: needs patch
2015-04-08 06:47:01python-devsetnosy: + python-dev
messages: + msg240251
2015-04-08 06:19:16berker.peksagsetnosy: + berker.peksag
2015-04-07 22:07:18martin.pantersetmessages: + msg240231
2015-04-07 16:23:31serhiy.storchakasetdependencies: + subprocess __all__ is incomplete
2015-04-07 16:23:10serhiy.storchakacreate