New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
built-in compile() should take encoding option. #50161
Comments
The built-in compile() expects source is encoded in utf-8. Below is current compile() behavior. # Python's interactive shell in Windows cp932 console.
>>> "あ"
'\x82\xa0'
>>> u"あ"
u'\u3042'
# compile() fails to decode str.
>>> code = compile('u"あ"', '__interactive__', 'single')
>>> exec code
u'\x82\xa0' # u'\u3042' expected.
# compile() encodes unicode to utf-8.
>>> code = compile(u'"あ"', '__interactive__', 'single')
>>> exec code
'\xe3\x81\x82' # '\x82\xa0' (cp932) wanted, but I get utf-8. Currentry, using PEP-0263 like below is needed to get compile >>> code = compile('# coding: cp932\n%s' % ('"あ"',), '__interactive__',
'single')
>>> exec code
'\x82\xa0'
>>> code = compile('# coding: cp932\n%s' % ('u"あ"',), '__interactive__',
'single')
>>> exec code
u'\u3042' But I feel compile() with PEP-0263 is bit dirty hack. Following example is describe behavior of compile() with encoding option. # coding: utf-8 (in utf-8 context)
code = compile('"あ"', '__foo.py', 'single')
exec code #=> '\xe3\x81\x82'
code = compile('"あ"', '__foo.py', 'single', encoding='cp932') =>
UnicodeDecodeError
code = compile(u'"あ"', '__foo.py', 'single')
exec code #=> '\xe3\x81\x82'
code = compile(u'"あ"', '__foo.py', 'single', encoding='cp932')
exec code #=> '\x82\xa0' |
add sample implementation. |
The patch as it currently stands is unacceptable because it changes |
Anyone interested in producing an updated patch? |
This problem is not heavy on Python 3. |
I'll close this then. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: