This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: warnings.warn fails silently with unicode input
Type: Stage: resolved
Components: Extension Modules Versions: Python 2.7
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: ZackerySpytz, nparslow, xtreak
Priority: normal Keywords:

Created on 2018-09-20 14:15 by nparslow, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg325880 - (view) Author: Nicholas Parslow (nparslow) Date: 2018-09-20 14:15

Python 2.7.14 |Anaconda, Inc.| (default, Dec  7 2017, 17:05:42) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import warnings
>>> warnings.warn(u'blé')
>>> warnings.warn('blah')
__main__:1: UserWarning: blah

the warnings.warn_explicit function seems to be the part that is failing. On a machine which doesn't use anaconda it fails in an identical manner.

if you do:
>>> with warnings.catch_warnings(record=True) as w:
...   warnings.warn(u'blé')
>>> w
[<warnings.WarningMessage object at 0x7ffadb35b190>]
>>> w[0].message

I can get that it fails with unicode (though it's annoying) but the silent fail is really bad imo.
msg326221 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-24 11:14
I think this a known limitation as per where non-ASCII characters would require byte string. I couldn't find the relevant documentation for it though.

$ ./python.exe
Python 2.7.15+ (remotes/upstream/2.7:69d0bc1430, Sep 24 2018, 16:02:09)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import warnings
>>> warnings.warn(b'blé')
__main__:1: UserWarning: blé
>>> warnings.warn(u'blé')

msg326222 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-24 11:30
I looked some more into the linked issue and I think this is an explicit decision and the comment in the code indicates that the warning will be lost : . I understand your perspective on failing silently but I am proposing to close this since enabling the error by removing UnicodeDecodeError in the except clause will be an incompatible change at this point.

msg326227 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-24 12:38
Since Nick had some issue on adding a message to the issue I am adding the email sent to me as a message below : 

(sorry I can't see how to post a reply message on the bugs page, I'm still new to this)

anyhow here's what I wanted to post:

Thanks for finding the bug!
Maybe a preferable behaviour would be to split the error types
 - an IOError could still pass (I'm really not sure what the ideal treatment in that case is
 - a Unicode error would still print a warning but it would be something like a generic 'warning: non-standard text found in warning text so only printing this' (I'm sure there's something better sounding / more consistent with other warnings but I don't know them that well)

so the code would look something like:
except (UnicodeError):
    message = "non-standard text found in warning text so only printing this"
    file.write(formatwarning(message, category, filename, lineno, line))
except (IOError):
        pass # the file (probably stderr) is invalid - this warning
 gets lost.

but I'm not sure if an IOError inside the UnicodeError handling would still be caught (?)

anyhow I'm not sure how standard/appropriate a generic warning would be within the broader context of python (?) I think for us we'll probably just bite the bullet and switch to python3 to avoid this kind of thing since we'll potentially have a lot of unicode messages in the near future.

thanks again,
Nick Parslow
msg384160 - (view) Author: Zackery Spytz (ZackerySpytz) * (Python triager) Date: 2021-01-01 11:01
Python 2.7 is no longer supported, so I think this issue should be closed.
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 78933
2021-01-01 12:27:40corona10setstatus: open -> closed
resolution: wont fix
stage: resolved
2021-01-01 11:01:01ZackerySpytzsetnosy: + ZackerySpytz
messages: + msg384160
2018-09-24 12:38:00xtreaksetmessages: + msg326227
2018-09-24 11:30:05xtreaksetmessages: + msg326222
2018-09-24 11:14:17xtreaksetmessages: + msg326221
2018-09-20 19:26:33xtreaksetnosy: + xtreak
2018-09-20 14:15:37nparslowcreate