This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: if a python program is execute by subprocess, the python program can't output unicode characters and raise UnicodeEncodeError
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.8
Status: closed Resolution: duplicate
Dependencies: Superseder: Encoding error running in subprocess with captured output
View: 34618
Assigned To: Nosy List: markhuang3310, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2022-03-11 16:41 by markhuang3310, last changed 2022-04-11 14:59 by admin. This issue is now closed.

File name Uploaded Description Edit markhuang3310, 2022-03-11 16:41 Repo file
Messages (3)
msg414919 - (view) Author: markhuang3310 (markhuang3310) Date: 2022-03-11 16:41
Windows 10
Python3.8.10 amd64


I try to get output from a python program with a subprocess.
The python program print a unicode character.

import subprocess
import sys
cmd = rf'''{sys.executable} -c "print('\u042f')"'''

print('case 1')
stdout, stderr =  subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=None).communicate()
print('case 2')
stdout, stderr =  subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()

The issue is that:
In case 1, I can see the "Я" (\u042f) form console.
In case 2, I try to get the output from subprocess.Popen(command).communicate()[0], the python process fail with   UnicodeEncodeError.

Expected result:
In case2, I should get stdout == b'\u042f'
msg414921 - (view) Author: markhuang3310 (markhuang3310) Date: 2022-03-11 16:43
the test output:

PS C:\case_dev> python3 .\
case 1
case 2
b'Traceback (most recent call last):\r\n  File "<string>", line 1, in <module>\r\n  File "C:\\Python36\\lib\\encodings\\", line 19, in encode\r\n    return codecs.charmap_encode(input,self.errors,encoding_table)[0]\r\nUnicodeEncodeError: \'charmap\' codec can\'t encode character \'\\u042f\' in position 0: character maps to <undefined>\r\n'
msg414924 - (view) Author: markhuang3310 (markhuang3310) Date: 2022-03-11 17:28
sorry, I found the duplication
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91144
2022-03-11 18:14:46eryksunsetsuperseder: Encoding error running in subprocess with captured output
2022-03-11 17:29:02markhuang3310setstatus: open -> closed
stage: resolved
2022-03-11 17:28:58markhuang3310setresolution: duplicate
messages: + msg414924
2022-03-11 16:43:29markhuang3310setmessages: + msg414921
2022-03-11 16:41:05markhuang3310create