Title: optparse module does not emit DeprecationWarning
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.6, Python 3.4, Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: John Hagen, SilentGhost, martin.panter, matrixise, r.david.murray, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-10-30 23:22 by John Hagen, last changed 2020-04-29 15:12 by John Hagen.

File name Uploaded Description Edit
optparse_and_test_deprecationwarning.patch John Hagen, 2015-11-01 20:27 Adds a PendingDeprecationWarning to optparse, and suppresses the warning in its tests. review
Messages (21)
msg253768 - (view) Author: John Hagen (John Hagen) * Date: 2015-10-30 23:22
According to the Python docs, optparse has been deprecated since Python 3.2, but it does not emit a DeprecationWarning when imported Python 3.2+.

PyCharm can uses these DeprecationWarnings to provide helpful alerts the user:
msg253779 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-10-31 08:06
Under <> it says “Importing optparse will issue a PendingDeprecationWarning” (not DeprecationWarning!), however even this does not appear to be the case. Maybe somebody forgot to do this, or maybe somebody decided not to do this; I dunno.
msg253788 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-31 14:23
Whether someone decided not to do it originally or not, it is probably a good idea to do it (as a DeprecationWarning, IMO), if someone wants to propose a patch.  The module isn't receiving maintenance any longer.  (Not that it couldn't, but no one is motivated to do so.)
msg253808 - (view) Author: John Hagen (John Hagen) * Date: 2015-10-31 21:10
First time trying to contribute to the Python standard library.

I attached a patch that is modeled off the deprecated imp module.  The DeprecationWarning message is taken from the Python docs:
msg253838 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-01 10:14
I don’t have an opinion on pending vs non-pending warning.

I think there are quite a few standard library modules that use “optparse” that will emit a warning as a result of this change. And the test suite probably needs patching to suppress the warning while testing the “optparse” module. Perhaps you could look at the code touched by revisions 5877c191b76e (imp warning upgrade) and 3f924cb4c3eb (test suite update) for inspiration.
msg253841 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2015-11-01 10:41
Modules that use optparse: calendar (issue18973 has patch), uu (seems trivial to change), test_decimal (trivial too), profile, cprofile

Tools: strginbench, ccbench, iobench

Neither of the changes are particularly tricky it seems.
msg253842 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2015-11-01 10:45
Already proposed to move to argparse for the existing modules, see this issue:

For the calendar module, there are two issues with two patches. and

I can write a patch to use argparse instead of optparse.
msg253857 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-01 15:48
There is a difference between optparse and other deprecated module. Other modules are deprecated if they are too buggy or bad-designed to be fixed and there are stable standard or third-party modules that provides similar functionality. But looking on a number of open issues for argparse it seems that optparse is more stable and has less bugs than more powerful argparse.

Having experience with converting many modules from optparse to argparse or reviewing converting patches, I can say that not always the conversion can be done without loss. For example there is a difference in formatting descriptions.

I think that optparse should be supported in the stdlib for very long time. -1 for non-pending warning. Don’t have an opinion about pending warning.
msg253858 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-01 16:02
And it looks there was a special decision to not deprecate optparse.
msg253869 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-01 19:38
My sense is that most of the open argparse bugs are *because* of the increased power (that is, they aren't, with a few exceptions, in the fundamental bits that optparse does as well).  And the issues are open because no one is maintaining that module, either :(.  There is one non-committer who has done a lot of work on the patches, we should probably evaluate offering him commit privileges.

That said, I don't have a strong opinion on the deprecation-or-not myself.
msg253874 - (view) Author: John Hagen (John Hagen) * Date: 2015-11-01 20:27
@martin.panter My new patch fixes Lib/ by suppressing the warning like test_imp does as you suggested.

@serhiy.storchaka I don't have a strong preference that it be a DeprecationWarning vs. PendingDeprecationWarning since to me, both get the point across to the end-user to avoid it.  I do, however, feel that it is important raise some kind of warning since it seems clear from the docs that is deprecated.

In my new patch I changed the type of warning to a PendingDeprecationWarning, as there seems to be at least some notion that optparse is a special case.  I don't have a strong opinion about which type of deprecation warning is used.

My new patch works when is run with -Werror on 3.6.0a0.
msg253881 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-01 22:21
Serhiy, do you have a link or more information about this decision? Looking around at previous discussions, this is what I found: limiting to pending deprecation: <>; not bothering with any code warning: <>. But the PEP was never updated as suggested in that last thread.

FWIW I have never used optparse. I do find argparse useful, but having bugs due to being “powerful” is not necessarily a good thing.
msg253884 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-01 23:07
No, I have not more information. My supposition is based on indirect evidence in msg225052. I suppose there was more recent discussion in 2014.
msg253889 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-02 01:24
I can't remember the timing...was that PEP accepted before we made DeprecationWarnings off by default?  It sounds like it probably was, based on the phrasing.
msg253890 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-02 01:28
Oh, and as far as bugs because it is powerful...optparse has had time to have its bugs fixed, so having bugs in the newer features of argparse is not surprising, really.  We just need to get the fixes committed.
msg254051 - (view) Author: John Hagen (John Hagen) * Date: 2015-11-04 12:04
Is there any consensus on how to move forward with this?  I feel there are at least 4 options:

1) Do nothing.
    Pro: No work.
    Con: It feels misleading to the user since the docs clearly state it's deprecated.  Some users (especially new ones) may miss the fact they should really be using argparse.

2) Throw PendingDeprecationWarning from optparse, and simply suppress it where it's used.
    Pro: Not as much work.  Users of optparse are properly notified.
    Con: Kicks the can down the road for someone else to have to eventually port off of optparse.

3) Throw PendingDeprecationWarning from optparse and port stdlib modules to argparse.
    Pro: Seems like the "purest" solution at least from what is indicated in docs that it will no longer be supported.  Users of optparse are properly notified.
    Con: Most amount of work.  argparse has some bugs that need to be patched.

4) Some combination of 2) and 3) where some modules are suppressed and others are ported.
msg254059 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-04 15:44
I vote for (4): (2) with a helping of (3).  That is, add the warning, but don't try to port all the stdlib modules to argparse beforehand.  Those should be addressed one by one, as I mentioned in another issue.
msg254061 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-04 16:06
I vote for gradual porting stdlib modules and scripts to argparse and fixing found problems, but doesn't touch optparse right now. We have enough time before 3.6 feature freezing.
msg254063 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2015-11-04 16:21
+1 with @serhiy.storchaka
msg265061 - (view) Author: John Hagen (John Hagen) * Date: 2016-05-07 11:52
With 3.6.0a1 scheduled just around the corner, is there consensus about how to begin this?  I just signed the contributor agreement, so I should be able to try to help if we have a plan on how to divide the work.
msg367655 - (view) Author: John Hagen (John Hagen) * Date: 2020-04-29 15:12
With PEP 594 ( in the pipeline, is it time that optparse correctly emits DeprecationWarnings?
Date User Action Args
2020-04-29 15:12:36John Hagensetmessages: + msg367655
2016-05-07 11:52:32John Hagensetmessages: + msg265061
2015-11-04 16:21:04matrixisesetmessages: + msg254063
2015-11-04 16:06:46serhiy.storchakasetmessages: + msg254061
2015-11-04 15:44:51r.david.murraysetmessages: + msg254059
2015-11-04 12:04:00John Hagensetmessages: + msg254051
2015-11-02 01:28:43r.david.murraysetmessages: + msg253890
2015-11-02 01:24:45r.david.murraysetmessages: + msg253889
2015-11-01 23:07:40serhiy.storchakasetmessages: + msg253884
2015-11-01 22:21:57martin.pantersetmessages: + msg253881
2015-11-01 20:28:02John Hagensetfiles: - optparse_deprecationwarning.patch
2015-11-01 20:27:41John Hagensetfiles: + optparse_and_test_deprecationwarning.patch

messages: + msg253874
2015-11-01 19:38:44r.david.murraysetmessages: + msg253869
2015-11-01 16:02:07serhiy.storchakasetmessages: + msg253858
2015-11-01 15:48:14serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg253857
2015-11-01 10:45:26matrixisesetnosy: + matrixise
messages: + msg253842
2015-11-01 10:41:47SilentGhostsetnosy: + SilentGhost
messages: + msg253841
2015-11-01 10:14:15martin.pantersetmessages: + msg253838
stage: patch review
2015-10-31 21:10:21John Hagensetfiles: + optparse_deprecationwarning.patch
keywords: + patch
messages: + msg253808
2015-10-31 14:23:26r.david.murraysetnosy: + r.david.murray
messages: + msg253788
2015-10-31 08:06:51martin.pantersetnosy: + martin.panter
messages: + msg253779
2015-10-30 23:22:34John Hagencreate