You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For my work on bpo-9425 (Rewrite import machinery to work with unicode paths), I need a PyArg_Parse converter converting bytes and str to str. PyUnicode_FSConverter() is the opposite because it encodes str to bytes.
To handle (input) filenames in a function, we have 3 choices:
1/ use bytes: that's the current choice for most Python functions. It gives full unicode support for POSIX OSes (FS using a bytes API), but it is not enough for Windows (Windows uses mbcs encoding which is a very small subset of Unicode)
2/ use str with the PEP-383 (surrogateescape): it begins to be used in Python 3.1, and more seriously in Python 3.2. It offers full unicode support on all OSes (POSIX and Windows)
3/ use the native type for each OS (bytes on POSIX, str on Windows): I dislike this solution because it implies code duplication
PyUnicode_FSConverter() is the converter for solution (1). PyUnicode_FSDecoder() will be the converter for the solution (2).
Lib/os.py may also be patched to add a Python implementation. Eg.
deffsdecode(value):
ifisinstance(value, str):
returnvalueelifisinstance(value, bytes):
encoding=sys.getfilesystemencoding()
ifencoding=='mbcs':
returnvalue.decode(encoding)
else:
returnvalue.decode(encoding, 'surrogateescape')
else:
raiseTypeError("expect bytes or str, not %s"%type(value).__name__)
--
Note: Solution (1) (use bytes API) is not deprecated by this issue. PyUnicode_FSConverter is still useful if the underlying library has a bytes API (eg. OpenSSL only supports char*).
Solution (2) is preferred if we have access to a character API, eg. Windows wide character API.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: