Message138484
> What is the use of these code_page_encode() functions?
I wrote them to be able to write tests.
We can maybe use them to implement the Python code page codecs using a
custom codec register function: see msg138246. Windows codecs seem to be
less reliable/portable than Python builtin codecs, they behave
differently depending on the Windows version. Windows codecs are maybe
faster, I should (write and) run a benchmark.
My main concern is to fix error handling of the Python mbcs codec.
--
I am also trying to factorize the code in posixmodule.c: I would like to
remove the bytes implementation of each function when a function has two
implementations (bytes and Unicode) only for Windows. The idea is to
decode filenames exactly as Windows do and reuse the Unicode
implementation. I don't know yet how Windows do decode bytes filenames
(especially how it handles undecodable bytes), I suppose that it uses
MultiByteToWideChar using cp=CP_ACP and flags=0.
We may patch os.fsdecode() to handle undecodable bytes like Windows
does. codecs.code_page_decode() would help this specific idea, except
that my current patch doesn't allow to specify directly the flags.
"replace" and "ignore" error handlers don't behave as flags=0, or at
least not in some cases. codecs.code_page_decode() should allow to
specific an error handler *or* the flags (mutual exclusive options).
Example:
def fsdecode(filename):
if isinstance(filename, bytes):
return codecs.code_page_decode(codecs.CP_ACP, filename, flags=0)
elif isinstance(filename, str):
return filename
else:
raise TypeError() |
|
Date |
User |
Action |
Args |
2011-06-17 00:35:35 | vstinner | set | recipients:
+ vstinner, loewis, amaury.forgeotdarc, ocean-city |
2011-06-17 00:35:34 | vstinner | link | issue12281 messages |
2011-06-17 00:35:33 | vstinner | create | |
|