This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients eryksun, ezio.melotti, jaraco, lemburg, loewis, r.david.murray, serhiy.storchaka, vstinner
Date 2020-10-12.12:08:07
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1602504488.02.0.550079592999.issue21927@roundup.psfhosted.org>
In-reply-to
Content
I'm closing this as a third-party issue with older versions of PowerShell. Newer versions of PowerShell set the output encoding to UTF-8 without a BOM preamble. For example:

    PS C:\> $PSVersionTable.PSVersion

    Major  Minor  Patch  PreReleaseLabel BuildLabel
    -----  -----  -----  --------------- ----------
    7      0      3

    PS C:\> $OutputEncoding.EncodingName
    Unicode (UTF-8)

    PS C:\> echo ¡¢£¤¥ | py -3 -X utf8 -c "print(ascii(input()))"
    '\xa1\xa2\xa3\xa4\xa5'

It's still possible to manually set the output encoding to include a BOM preamble. For example:

    PS C:\> $OutputEncoding = [System.Text.Encoding]::UTF8
    PS C:\> $OutputEncoding.GetPreamble()
    239
    187
    191
    PS C:\> echo ¡¢£¤¥ | py -3 -X utf8 -c "print(ascii(input()))"
    '\ufeff\xa1\xa2\xa3\xa4\xa5'

I don't know what would be appropriate for Python's I/O stack in terms of detecting and handling a UTF-8 preamble on any type of file (console/terminal, pipe, disk), i.e. using the "utf-8-sig" encoding instead of "utf-8", as opposed to just letting scripts detect and handle an initial BOM character (U+FEFF) however they see fit. But that discussion needs a new issue if people are interested in supporting new behavior.
History
Date User Action Args
2020-10-12 12:08:08eryksunsetrecipients: + eryksun, lemburg, loewis, jaraco, vstinner, ezio.melotti, r.david.murray, serhiy.storchaka
2020-10-12 12:08:08eryksunsetmessageid: <1602504488.02.0.550079592999.issue21927@roundup.psfhosted.org>
2020-10-12 12:08:08eryksunlinkissue21927 messages
2020-10-12 12:08:07eryksuncreate