This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ncoghlan
Recipients Balthazar.Rouberol, antlong, barry, docs@python, eric.araujo, ezio.melotti, georg.brandl, hhas, jleedev, kousu, ncoghlan, pitrou, r.david.murray, serhiy.storchaka
Date 2013-11-30.14:06:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1385820361.46.0.742716891009.issue10976@psf.upfronthosting.co.za>
In-reply-to
Content
Issue 19837 is the complementary problem on the serialisation side - users migrating from Python 2 are accustomed to being able to use the json module directly as a wire protocol module, but the strict Python 3 interpretation as a text transform means that isn't possible - you have to apply the text encoding step separately.

What appears to have happened is that the way JSON is used in practice has diverged from JSON as a formal spec.

Formal spec (this is what the Py3k JSON module implements, and Py2 implements with ensure_ascii=False): JSON is a Unicode text transform, which may optionally be serialised as UTF-8, UTF-16 or UTF-32.

Practice (what the Py2 JSON module implements with ensure_ascii=True, and what is covered in RFC 4627): JSON is a UTF-8 encoded wire protocol

So now we're left with the options:

- try to tweak the existing json APIs to handle both the str<->str and str<->bytes use cases (ugly)
- add new APIs within the existing json module
- add a new "jsonb" module, which dumps to UTF-8 encoded bytes, and reads from UTF-8, UTF-16 or UTF-32 encoded bytes in accordance with RFC 4627 (but being more tolerant in terms of what is allowed at the top level)

I'm currently leaning towards the "jsonb" module option, and deprecating the "encoding" argument in the pure text version. It's not pretty, but I think it's better than the alternatives.
History
Date User Action Args
2013-11-30 14:06:01ncoghlansetrecipients: + ncoghlan, barry, georg.brandl, hhas, pitrou, kousu, ezio.melotti, eric.araujo, r.david.murray, docs@python, antlong, serhiy.storchaka, Balthazar.Rouberol, jleedev
2013-11-30 14:06:01ncoghlansetmessageid: <1385820361.46.0.742716891009.issue10976@psf.upfronthosting.co.za>
2013-11-30 14:06:01ncoghlanlinkissue10976 messages
2013-11-30 14:06:01ncoghlancreate