classification
Title: PEP 597: netrc uses locale encoding.
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: methane, xiang.zhang
Priority: normal Keywords: patch

Created on 2021-04-05 12:46 by methane, last changed 2021-05-02 05:01 by methane. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25781 merged methane, 2021-05-01 14:02
Messages (3)
msg390232 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-04-05 12:46
https://github.com/python/cpython/blob/c8e5eb904e12010d2302364e1037c24a30f5e241/Lib/netrc.py#L29

Can we change the encoding="utf-8" and errors="replace"?

IMHO, comments are source of UnicodeDecodeError. So we can open file with binary mode and skip comments without decoding it.
But we need to decode non-comment lines lines with some encoding anyway.
msg391043 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-04-14 08:11
I googled "netrc UnicodeDecodeError". It is very rare, but I found two cases.

* https://stackoverflow.com/questions/54748450/macos-python-3-netrc-operations-end-up-with-unicodedecodeerror
  This user uses macOS and Python tried UTF-8. But .netrc was not UTF-8.

* https://qiita.com/yuji38kwmt/items/5472c98cb800ccaab093
  This user uses Windows. They write .netrc with UTF-8, but Python used cp932.

---

There are several options:

* Use UTF-8. If user encountered UnicodeDecodeError, user need to change the .netrc encoding.
* Use UTF-8 with "surrogateescape" or "replace" error handler. non-UTF-8 in comments are ignored.
* Use latin-1. non-ASCII characters in comments are ignored.
msg392667 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-05-02 05:01
New changeset fd0bc7e7f4f2c7de98a1ebc7ad1ef65b8f8f7ad6 by Inada Naoki in branch 'master':
bpo-43733: netrc try to use UTF-8 before using locale encoding. (GH-25781)
https://github.com/python/cpython/commit/fd0bc7e7f4f2c7de98a1ebc7ad1ef65b8f8f7ad6
History
Date User Action Args
2021-05-02 05:01:22methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-05-02 05:01:18methanesetmessages: + msg392667
2021-05-01 14:02:09methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request24472
2021-04-14 08:11:15methanesetmessages: + msg391043
2021-04-14 05:18:49methanesetnosy: + xiang.zhang
2021-04-05 12:46:50methanecreate