classification
Title: Plistlib outputs empty data tags when deeply nested
Type: behavior Stage: needs patch
Components: Library (Lib), Macintosh Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: ezio.melotti, jfortier, mmilkin, python-dev, ronaldoussoren, serhiy.storchaka
Priority: normal Keywords: easy

Created on 2013-03-04 20:33 by jfortier, last changed 2013-04-23 11:51 by ronaldoussoren. This issue is now closed.

Files
File name Uploaded Description Edit
deeply-nested-plist.py ronaldoussoren, 2013-04-15 12:45
issue-17353.txt ronaldoussoren, 2013-04-15 12:46 review
issue-17353-tests.txt mmilkin, 2013-04-17 00:37 review
Messages (13)
msg183496 - (view) Author: Jacques Fortier (jfortier) Date: 2013-03-04 20:33
To reproduce:
import plistlib
print plistlib.writePlistToString([[[[[[[[[{'test': plistlib.Data('aaaaaa')}]]]]]]]]])

Compare to:
print plistlib.writePlistToString([[[[[[[[{'test': Data('aaaaaa')}]]]]]]]])

It looks like the max line length calculation in the Data class is causing the output to be empty once you get nested far enough.
msg183497 - (view) Author: Jacques Fortier (jfortier) Date: 2013-03-04 20:36
My second print should have been:

print plistlib.writePlistToString([[[[[[[[{'test': plistlib.Data('aaaaaa')}]]]]]]]])
msg183509 - (view) Author: Jacques Fortier (jfortier) Date: 2013-03-05 05:15
Quick source inspection suggests this is present in the latest version.
msg183525 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-03-05 13:34
We should limit the indentation to be less than the maximal line length.
msg186929 - (view) Author: Mike Milkin (mmilkin) * Date: 2013-04-14 17:07
Looks like plistlib.writePlistToString is no loger in the plistlib.
msg186941 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-04-14 18:06
On 3.x it's called writePlistToBytes, and it seems to have the same issue.
msg186944 - (view) Author: Mike Milkin (mmilkin) * Date: 2013-04-14 18:57
Yep you are correct.
>>>
plistlib.readPlistFromBytes(plistlib.writePlistToBytes([[[[[[[[{'test': plistlib.Data(b'aaaaaa')}]]]]]]]]))
[[[[[[[[{'test': Data(b'aaaaaa')}]]]]]]]]
>>> plistlib.readPlistFromBytes(plistlib.writePlistToBytes([[[[[[[[[{'test': plistlib.Data(b'aaaaaa')}]]]]]]]]]))
[[[[[[[[[{'test': Data(b'')}]]]]]]]]]

I presume we want to raise a ValueError if the indentation is more than the maximal line length.
msg186967 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-04-15 03:26
If possible it would be better to still generate a valid output, possibly by limiting the indentation or removing it after a certain point.
However I don't know if there are any restrictions that should be followed while generating a plist (What happens if the line is too long? Is the indentation required?).
msg186981 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-04-15 12:45
I agree that plistlib shouldn't raise an exception for data that can represented as a valid plist file.

I've checked that the Cocoa class for generating plist files will happily create a plist file when the data is nested 100 levels deep. In that case NSData values generate lines of 12 characters long.



An unrelated issue: PlistWriter.writeValue should treat bytes instances the same as Data instances in Python 3. That would be a (small) feature enhencement, and hence can only be done for Python 3.4.
msg186982 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-04-15 12:46
The attach patch should fix the issue (but there needs to be a unittest as well).
msg187127 - (view) Author: Mike Milkin (mmilkin) * Date: 2013-04-17 00:37
Adding tests.
msg187153 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-04-17 10:45
The tests look good, thanks for writing them.
msg187629 - (view) Author: Roundup Robot (python-dev) Date: 2013-04-23 11:50
New changeset a389700fbc74 by Ronald Oussoren in branch '2.7':
Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/a389700fbc74

New changeset 6a8cb4875ac6 by Ronald Oussoren in branch '3.3':
Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/6a8cb4875ac6

New changeset e0c0bcd60033 by Ronald Oussoren in branch 'default':
(3.3->default) Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/e0c0bcd60033
History
Date User Action Args
2013-04-23 11:51:27ronaldoussorensetstatus: open -> closed
resolution: fixed
2013-04-23 11:50:53python-devsetnosy: + python-dev
messages: + msg187629
2013-04-17 10:45:22ronaldoussorensetmessages: + msg187153
2013-04-17 00:37:02mmilkinsetfiles: + issue-17353-tests.txt

messages: + msg187127
2013-04-15 12:46:24ronaldoussorensetfiles: + issue-17353.txt

messages: + msg186982
2013-04-15 12:45:19ronaldoussorensetfiles: + deeply-nested-plist.py

messages: + msg186981
2013-04-15 03:26:44ezio.melottisetmessages: + msg186967
2013-04-14 18:57:31mmilkinsetmessages: + msg186944
2013-04-14 18:06:12ezio.melottisetmessages: + msg186941
versions: - Python 3.2
2013-04-14 17:07:13mmilkinsetnosy: + mmilkin
messages: + msg186929
2013-03-11 20:05:22ezio.melottisetkeywords: + easy
nosy: + ezio.melotti
2013-03-05 13:34:04serhiy.storchakasetassignee: ronaldoussoren
type: behavior
components: + Macintosh
versions: + Python 3.2, Python 3.3, - Python 3.5
nosy: + serhiy.storchaka, ronaldoussoren

messages: + msg183525
stage: needs patch
2013-03-05 05:15:17jfortiersetmessages: + msg183509
versions: + Python 3.4, Python 3.5
2013-03-04 20:36:20jfortiersetmessages: + msg183497
2013-03-04 20:33:35jfortiercreate