Message378486
I'm closing this as a third-party issue with older versions of PowerShell. Newer versions of PowerShell set the output encoding to UTF-8 without a BOM preamble. For example:
PS C:\> $PSVersionTable.PSVersion
Major Minor Patch PreReleaseLabel BuildLabel
----- ----- ----- --------------- ----------
7 0 3
PS C:\> $OutputEncoding.EncodingName
Unicode (UTF-8)
PS C:\> echo ¡¢£¤¥ | py -3 -X utf8 -c "print(ascii(input()))"
'\xa1\xa2\xa3\xa4\xa5'
It's still possible to manually set the output encoding to include a BOM preamble. For example:
PS C:\> $OutputEncoding = [System.Text.Encoding]::UTF8
PS C:\> $OutputEncoding.GetPreamble()
239
187
191
PS C:\> echo ¡¢£¤¥ | py -3 -X utf8 -c "print(ascii(input()))"
'\ufeff\xa1\xa2\xa3\xa4\xa5'
I don't know what would be appropriate for Python's I/O stack in terms of detecting and handling a UTF-8 preamble on any type of file (console/terminal, pipe, disk), i.e. using the "utf-8-sig" encoding instead of "utf-8", as opposed to just letting scripts detect and handle an initial BOM character (U+FEFF) however they see fit. But that discussion needs a new issue if people are interested in supporting new behavior. |
|
Date |
User |
Action |
Args |
2020-10-12 12:08:08 | eryksun | set | recipients:
+ eryksun, lemburg, loewis, jaraco, vstinner, ezio.melotti, r.david.murray, serhiy.storchaka |
2020-10-12 12:08:08 | eryksun | set | messageid: <1602504488.02.0.550079592999.issue21927@roundup.psfhosted.org> |
2020-10-12 12:08:08 | eryksun | link | issue21927 messages |
2020-10-12 12:08:07 | eryksun | create | |
|