Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Plistlib: Half of the double width characters are missing when writing binary plist #72508

Closed
kuglee mannequin opened this issue Sep 30, 2016 · 3 comments
Closed

Plistlib: Half of the double width characters are missing when writing binary plist #72508

kuglee mannequin opened this issue Sep 30, 2016 · 3 comments
Assignees
@serhiy-storchaka
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@kuglee
Copy link
Mannequin

kuglee mannequin commented Sep 30, 2016

BPO 28321
Nosy @ronaldoussoren, @serhiy-storchaka
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
    		•
    Files
  • input.plist: This is the input file
  • output.plist: This is the output file.
  • plistlib-astral-characters.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-10-04.17:10:19.838>
created_at = <Date 2016-09-30.22:57:38.388>
labels = ['3.7', 'type-bug', 'library']
title = 'Plistlib: Half of the double width characters are missing when writing binary plist'
updated_at = <Date 2017-03-31.16:36:39.033>
user = 'https://bugs.python.org/kuglee'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:39.033>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-10-04.17:10:19.838>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2016-09-30.22:57:38.388>
creator = 'kuglee'
dependencies = []
files = ['44897', '44898', '44901']
hgrepos = []
issue_num = 28321
keywords = ['patch']
message_count = 3.0
messages = ['277781', '277800', '278062']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'python-dev', 'serhiy.storchaka', 'kuglee']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28321'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
    @kuglee kuglee mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 30, 2016
    @kuglee
    Copy link
    Mannequin Author

    kuglee mannequin commented Sep 30, 2016

    I read an emoji character from a plist file. The emoji printed correctly to stdout. However when I dump the file to a binary plist only the half of the emoji was present.

    @serhiy-storchaka
    Copy link
    Member

    The simplest reproducer:

    >>> import plistlib
>>> plistlib.loads(plistlib.dumps('\U0001f40d', fmt=plistlib.FMT_BINARY))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/plistlib.py", line 1006, in loads
    fp, fmt=fmt, use_builtin_types=use_builtin_types, dict_type=dict_type)
  File "/usr/lib/python3.5/plistlib.py", line 997, in load
    return p.parse(fp)
  File "/usr/lib/python3.5/plistlib.py", line 623, in parse
    return self._read_object(self._object_offsets[top_object])
  File "/usr/lib/python3.5/plistlib.py", line 704, in _read_object
    return self._fp.read(s * 2).decode('utf-16be')
  File "/usr/lib/python3.5/encodings/utf_16_be.py", line 16, in decode
    return codecs.utf_16_be_decode(input, errors, True)
UnicodeDecodeError: 'utf-16-be' codec can't decode bytes in position 0-1: unexpected end of data

    Proposed patch fixes this issue.

    @serhiy-storchaka serhiy-storchaka added the 3.7 (EOL) end of life label Oct 1, 2016
    @serhiy-storchaka serhiy-storchaka self-assigned this Oct 4, 2016
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 4, 2016

    New changeset 381ef0f08f89 by Serhiy Storchaka in branch '3.5':
    Issue bpo-28321: Fixed writing non-BMP characters with binary format in plistlib.
    https://hg.python.org/cpython/rev/381ef0f08f89

    New changeset 3a7234d04fe9 by Serhiy Storchaka in branch '3.6':
    Issue bpo-28321: Fixed writing non-BMP characters with binary format in plistlib.
    https://hg.python.org/cpython/rev/3a7234d04fe9

    New changeset b6c85e7e558a by Serhiy Storchaka in branch 'default':
    Issue bpo-28321: Fixed writing non-BMP characters with binary format in plistlib.
    https://hg.python.org/cpython/rev/b6c85e7e558a

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @serhiy-storchaka