This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Aaron.Staley
Recipients Aaron.Staley
Date 2012-04-13.22:10:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1334355033.57.0.314676653112.issue14573@psf.upfronthosting.co.za>
In-reply-to
Content
The json library's encoder includes a function called 'iterencode'.  iterencode allows for encoding to be streamed; as tokens are produced they are yielded. This allows for the encoded object to be streamed to a file, over a socket, etc. without being placed all into memory.

Unfortunately, iterencode cannot encode general iterators.  This significantly limits the usefulness of the function.  For my use case I wish to convert a large stream (iterator) of objects into json.  Unfortunately, I currently have to:

A. Bring all the objects into memory by encasing the iterator in a list()
B. Make a hack where I subclass list and making that object's __iter__ function return my desired iterator.

The problem is that the json library explicitly checks for something being a list:

                if isinstance(value, (list, tuple)):
                    chunks = _iterencode_list(value, _current_indent_level)

It would work just as well (and be more pythonic) to see if the value supports the iterator protocol:
                if isinstance(value, collections.Iterable):
                    chunks = _iterencode_list(value, _current_indent_level)


Erroring example:

>>> import json
>>> e = json.JSONEncoder()
>>> r = xrange(20)
>>> gen = e.iterencode(r)
<generator object _iterencode at 0x14a5460>
>>> next(gen)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/json/encoder.py", line 419, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.2/json/encoder.py", line 170, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: xrange(0, 20) is not JSON serializable
History
Date User Action Args
2012-04-13 22:10:33Aaron.Staleysetrecipients: + Aaron.Staley
2012-04-13 22:10:33Aaron.Staleysetmessageid: <1334355033.57.0.314676653112.issue14573@psf.upfronthosting.co.za>
2012-04-13 22:10:33Aaron.Staleylinkissue14573 messages
2012-04-13 22:10:32Aaron.Staleycreate