Message393868
I've experienced a UnicodeDecodeError when adding unicode strings that contain multibye utf-8 characters into a shareable list.
My observation is that ShareableList chunks the list of strings before sending it over the process boundary, however this chunking process is not multibyte aware and will chunk in the middle of multibyte characters.
On the other end, this results in the ShareableList throwing a UnicodeDecodeError when it fails to decode not-a-full multibyte utf-8 character.
From running the attached MWE, I see that the string is sent in two chunks, the first being b'Boom \xf0\x9f\x92\xa5 \xf0\x9f\x92\xa5 \xf0' which clearly splits the 4 bytes of the 💥 character into the first byte and remaining 3 bytes. |
|
Date |
User |
Action |
Args |
2021-05-18 11:12:09 | huwcbjones | set | recipients:
+ huwcbjones |
2021-05-18 11:12:09 | huwcbjones | set | messageid: <1621336329.38.0.49764469303.issue44170@roundup.psfhosted.org> |
2021-05-18 11:12:09 | huwcbjones | link | issue44170 messages |
2021-05-18 11:12:09 | huwcbjones | create | |
|