Title: -W option cannot use non-standard categories
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Gerrit.Holl, THRlWiTi, brett.cannon, kernc, mitya57, pitrou, remram, torsten
Priority: normal Keywords:

Created on 2014-10-02 21:32 by remram, last changed 2018-11-29 16:40 by kernc.

Messages (6)
msg228261 - (view) Author: Rémi Rampin (remram) * Date: 2014-10-02 21:32
warnings._processoptions is called very early, before site-packages are enabled. Because of this, using a non-standard 'category' will almost certainly fail with the message:

    Invalid -W option ignored: invalid module name: '...'

The -W option would be a lot more useful if it could actually match non-standard categories (it does, after all, pretend to support modulename.classname).

I don't see any easy way of fixing this, other than initializing the warnings module later or matching category names with the given string (and getting rid of the import).
msg228286 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-03 00:54
How would this work? Would it auto-import the module?
msg228287 - (view) Author: Rémi Rampin (remram) * Date: 2014-10-03 01:01
It already does auto-import, but it does it before site-packages are set up, meaning that it fails in any practical setup.

See _getcategory(), called by _processoptions():
msg228326 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2014-10-03 12:58
Doing the initialization later is not really an option as there can be warnings in the parser so it has to be resolved very early on.

I guess some form could be a possibility, but that's also tricky as subclass matching also still needs to work. I think we would need to see a patch on the proposed solution before considering its acceptance.

Another option would to parse it twice: once early on for Python internals and then again later once site-packages and various other things have been set up. Once again I would need to look at a patch before making a decision on whether it's an acceptable solution.
msg276416 - (view) Author: Torsten Landschoff (torsten) * Date: 2016-09-14 11:18
Wow, this was news to me and I just ran into it in python 2.7. Checked in Python 3 and it's still there:

(py3)->torsten.landschoff@horatio:~$ python3 --version
Python 3.6.0a3+
(py3)->torsten.landschoff@horatio:~$ python3 -W error::sqlalchemy.exc.SAWarning -c "print()"
Invalid -W option ignored: invalid module name: 'sqlalchemy.exc'


I see no easy way to fix this. One way I thought about is to have hooks for importing of modules (called when a module is put into ``sys.modules``, not sure if there is something like this already) and have the warning system trigger on that.

There is no way to raise an exception before it is imported anyway...
msg330700 - (view) Author: Gerrit Holl (Gerrit.Holl) * Date: 2018-11-29 16:16
There is a thread on StackOverflow related to this problem:

The (currently only and accepted) answer proposes as a workaround to set the PYTHONPATH variable, as the contents of those apparently *are* already in sys.path at the time warnings._processoptions is called, unlike site-packages.  So manually adding site-packages to the PYTHONPATH variable does circumvent this problem (tested in Python 3.7.1).
Date User Action Args
2018-11-29 16:40:36kerncsetnosy: + kernc
2018-11-29 16:16:45Gerrit.Hollsetnosy: + Gerrit.Holl

messages: + msg330700
versions: + Python 3.6, Python 3.7
2018-11-12 19:28:45mitya57setnosy: + mitya57
2018-09-07 12:24:24THRlWiTisetnosy: + THRlWiTi
2016-09-14 11:18:12torstensetnosy: + torsten
messages: + msg276416
2014-10-03 12:58:38brett.cannonsetmessages: + msg228326
versions: - Python 3.1, Python 2.7, Python 3.2, Python 3.3, Python 3.4
2014-10-03 01:01:09remramsetmessages: + msg228287
2014-10-03 00:54:34pitrousetnosy: + brett.cannon, pitrou
messages: + msg228286
2014-10-02 21:32:00remramcreate