classification
Title: os.listdir documentation should mention surrogateescape
Type: Stage: resolved
Components: Documentation Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eric.araujo, iritkatriel, loewis, michael.foord, misdre, vstinner
Priority: normal Keywords: patch

Created on 2011-12-09 01:49 by michael.foord, last changed 2020-11-16 21:28 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
listdir-pep383.patch misdre, 2011-12-17 18:07 mention surrogateescape and pep 383 review
Messages (6)
msg149070 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2011-12-09 01:49
Where os.listdir encounters undecodable bytes from the filesystem it uses the surrogateescape handler. As the resulting strings are invalid they can't be encoded without an errorhandler, and so can't be printed (for example).

This should be documented.
msg149264 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-12-12 01:09
Can you please write a doc patch?
msg149686 - (view) Author: (misdre) Date: 2011-12-17 18:07
Added a small patch to mention surrogateescape and PEP 383.
msg150466 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-02 17:16
Patch looks good, apart from a missing “the”, but I think it should be expanded: Right now it tells that surrogateescape is used, but IMO it would be more useful if it also mentioned practical implications, i.e. what Michael “The Beard” Foord says in the first message.
msg381129 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-16 17:27
I think this was added in another section of this doc. See the second paragraph in 
https://docs.python.org/3/library/os.html#file-names-command-line-arguments-and-environment-variables
msg381161 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-16 21:28
> I think this was added in another section of this doc. See the second paragraph in 
> https://docs.python.org/3/library/os.html#file-names-command-line-arguments-and-environment-variables

Right, this section now explains encodings and the error handler.

Moreover, I recently reworked the documentation of the filesystem encoding and error handler:
* https://docs.python.org/dev/glossary.html#term-filesystem-encoding-and-error-handler
* https://docs.python.org/dev/library/os.html#python-utf-8-mode
* https://docs.python.org/dev/library/sys.html#sys.getfilesystemencoding
* etc.

I consider that it's now enough, I close the issue ;-) If someone wants to enhance the doc, please submit a PR. This issue is now quite old :-)
History
Date User Action Args
2020-11-16 21:28:10vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg381161

stage: needs patch -> resolved
2020-11-16 17:27:56iritkatrielsetnosy: + iritkatriel
messages: + msg381129
2012-01-02 17:16:48eric.araujosetmessages: + msg150466
2011-12-17 18:07:25misdresetfiles: + listdir-pep383.patch

nosy: + misdre
messages: + msg149686

keywords: + patch
2011-12-12 01:09:03vstinnersetmessages: + msg149264
2011-12-10 16:34:00eric.araujosetnosy: + loewis, vstinner, eric.araujo
2011-12-09 01:49:59michael.foordcreate