classification
Title: Suboptimal stacklevel of deprecation warnings for formatter and imp modules
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: duplicate
Dependencies: Superseder: The new import system makes it inconvenient to correctly issue a deprecation warning for a module
View: 24305
Assigned To: brett.cannon Nosy List: Arfrever, berker.peksag, brett.cannon, eric.snow, larry, ncoghlan, serhiy.storchaka
Priority: deferred blocker Keywords: patch

Created on 2015-03-30 01:54 by Arfrever, last changed 2015-08-14 18:18 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
better_stacklevel.diff brett.cannon, 2015-04-24 15:14 review
deprecated_module_stacklevel.diff brett.cannon, 2015-05-29 17:40 review
Messages (11)
msg239554 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2015-03-30 01:54
https://hg.python.org/cpython/rev/2a336cc29282 changed stacklevel of some deprecation warnings.
However new value is still not useful, because either _frozen_importlib or importlib/_bootstrap.py is now mentioned in deprecation warnings:

$ cat test.py
import formatter
import imp
$ python3.4 -Wd test.py
/usr/lib64/python3.4/formatter.py:24: PendingDeprecationWarning: the formatter module is deprecated and will be removed in Python 3.6
  'Python 3.6', PendingDeprecationWarning)
/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  PendingDeprecationWarning)
$ python3.5 -Wd test.py
_frozen_importlib:321: DeprecationWarning: the formatter module is deprecated and will be removed in Python 3.6
/usr/lib64/python3.5/importlib/_bootstrap.py:321: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  return f(*args, **kwds)
msg239607 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-03-30 13:33
Probably need to introduce a new keyword argument just for deprecated imports or some helper function in importlib to get the stack depth right (else there is a risk of breaking the stack depth in any minor release whenever importlib's depth shifts). Something like the following should be enough (obviously done in warnings instead of per-file):

try:
    level = 1
    while True:
        frame = sys._getframe(level)
        print(frame.f_code.co_filename)
        if '_bootstrap' not in frame.f_code.co_filename:
            break
        level += 1
except ValueError:
    pass
print(sys._getframe(2).f_code.co_filename)
warnings.warn("the imp module is deprecated in favour of importlib; "
              "see the module's documentation for alternative uses",
              PendingDeprecationWarning, stacklevel=level+1)

Otherwise the depths should just go back to what they were at.
msg239610 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-30 13:53
Similar feature is needed for warnings in the re module. Methods of regular expression pattern can be called directly or from module-level wrappers. In these case the stack level differs by 1. And sometimes warnings are emitted in recursive parser, when stack level is variable.
msg241944 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-04-24 15:14
Here is a private function in warnings for calculating the stack depth to the first frame not referencing some key string. Can someone look at it to make sure it looks reasonable?

I'm starting with it being private since it uses sys._getframe() and I don't know how widely needed it is.
msg241968 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-24 18:20
Unfortunately this will not help for re, because the trace passes through three files: Lib/sre_parse.py, Lib/sre_compile.py and Lib/re.py.
msg242015 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-04-25 14:38
OK, I'll look at making it more general for multiple names to skip.
msg244260 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-05-28 04:04
Issue 24305 covers either making this a public API for general use, or else just making module level deprecation warnings skip the import machinery automatically.

I also wonder whether Eric's _boostrap_external changes might have broken any of the frame hiding tricks for tracebacks.
msg244304 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-05-28 15:07
I had a similar concern, Nick, but don't think I did anything that would have broken the frame hiding logic.  That said, I did not take stacklevel for warnings into account.
msg244396 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-05-29 17:40
Latest patch should work for Serhiy's needs by taking a container of names to compare against the filename instead of a single argument.
msg246285 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-07-05 01:52
This regression isn't thrilling, but it's not the kind of "OMG we can't release with this bug" level of escalation I associate with an actual release blocker.  Let's at least defer it for now, and maybe we'll even reduce it further later.
msg248600 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-08-14 18:18
Merging with the other issue so there is a single place to track this
History
Date User Action Args
2015-08-14 18:30:49brett.cannonunlinkissue24305 dependencies
2015-08-14 18:18:57brett.cannonsetsuperseder: Suboptimal stacklevel of deprecation warnings for formatter and imp modules -> The new import system makes it inconvenient to correctly issue a deprecation warning for a module
2015-08-14 18:18:57brett.cannonunlinkissue23810 superseder
2015-08-14 18:18:33brett.cannonsetstatus: open -> closed
resolution: duplicate
2015-08-14 18:18:23brett.cannonsetsuperseder: Suboptimal stacklevel of deprecation warnings for formatter and imp modules
messages: + msg248600
2015-08-14 18:18:23brett.cannonlinkissue23810 superseder
2015-07-05 01:52:49larrysetpriority: release blocker -> deferred blocker

messages: + msg246285
2015-05-29 17:41:03brett.cannonsetversions: + Python 3.6
2015-05-29 17:40:56brett.cannonsetfiles: + deprecated_module_stacklevel.diff

messages: + msg244396
2015-05-28 15:07:12eric.snowsetnosy: + eric.snow
messages: + msg244304
2015-05-28 04:04:42ncoghlansetnosy: + ncoghlan
messages: + msg244260
2015-05-28 04:02:49ncoghlanlinkissue24305 dependencies
2015-04-25 14:38:49brett.cannonsetmessages: + msg242015
2015-04-24 18:20:48serhiy.storchakasetmessages: + msg241968
2015-04-24 15:14:31brett.cannonsetfiles: + better_stacklevel.diff
versions: - Python 3.4
messages: + msg241944

keywords: + patch
stage: test needed -> patch review
2015-03-30 18:35:33berker.peksagsetnosy: + berker.peksag
2015-03-30 13:53:49serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg239610
2015-03-30 13:33:21brett.cannonsetpriority: normal -> release blocker

nosy: + larry
messages: + msg239607

type: behavior
stage: test needed
2015-03-30 01:54:14Arfrevercreate