The Python tutorial tells, and even recommends, new users that they can use the pickle module to serialize arbitrary objects. However it does not provide any warning about the insecurity of unpickling arbtirary data. The text even goes so far as to mention sending pickled data over a network connection to other machines.

I believe this section should be replaced with using the json module instead of pickle. It is more standard and doesn't present the same security concerns with untrusted data as pickle does. However if it continues to recommend pickle to new users it should at least warn them of the dangers of using pickle.

The section in question is located at http://docs.python.org/3/tutorial/inputoutput.html#the-pickle-module

Advising the reader to be aware of the security warnings in the API documentation seems sufficient.

JSON isn't intended to support arbitrary data, and that's what this section is discussing.  Another section about data interchange with other applications (regardless of language), may be a reasonable addition, or a good candidate for a separate How-To document that can be referenced.

The section to me just seems to be about how to handle more than just strings, it mentions numbers, lists, dictionaries, and class instances. Of those it mentions, only the class instances are not able to handled out of the box by JSON.

However like I said even if it remains pickle this particular area of the documentation should still warn users even though there's already a warning in the API documentation for pickle. As it is if a new user reads this and doesn't click through to the API documentation they've received recommendation from the Python documentation that they can send pickle strings over the network. This is dangerous behavior and the documentation shouldn't be advising new users to do dangerous things by default.

Further more the tutorial claims it's the standard way of persisting data which in my experience it's far from that due to the security concerns. I've seen very little actual use of pickle in the wild (and when it was used it was often used by people who didn't understand the security implications).

When I read "... that can take almost any Python object  ...", I don't think the recommendation is about just a few types.

The Zope and ZODB communities certainly use pickle extensively, we're aware of the security implications, and we send pickled data over the network all the time.

A description of the pickle module itself does not equate to the purpose of the section. Given that this is a tutorial and previous section taught how to read and write from files I would suggest that the purpose of the section was to give them the next step to persisting data which could be pickle or it could be JSON (or it could be another format all together). 

I don't see what Zope/ZODB's awareness of the security implications has to do with anything unless you're trying to state that the developers and users of ZODB are newcomers to Python whose knowledge of pickle stems from what they read in the tutorial. However I am glad that those communities are aware of the implications if they are using that module, but the point is the reader of the tutorial is *not* likely to be aware of them and should *not* be using pickle without being aware of them especially if they are sending that data over the network.

I'm really not sure what your problem is here. What is there to lose by annotating this section of the tutorial with a similar warning as exists in the pickle documentation that they should not unpickle data from an untrusted or unauthenticated sources? I can see how someone could state that the problem with switching to JSON is that it's harder to construct a serialization for arbitrary objects. That's not something I think it all that important to teach someone brand new to Python however I can understand if switching this section to using a "safe" serialization format doesn't sit well with people which is why I suggested at least adding a warning.

So what exactly is your problem with at a minimum adding a warning?

I would be ok with changing that part of the tutorial to use json. Since json is much better known outside of the Python programming circles, and since its output is human-readable, it's probably a better fit for the tutorial. pickle can be mentioned as a more powerful and more dangerous alternative.

By the way, there is one difference between json and pickle in this context: json will output a text serialization, pickle a binary one. If serializing to a binary file, users must do the (utf-8, most likely) encoding themselves.

For the tutorial, I agree with presenting json rather than pickle for all the reasons given, with pickle mentioned in a paragraph at the end (more powerful, more dangerous, see warning in manual before using).

How about we simply add the warning from http://docs.python.org/3/library/pickle.html#module-pickle to the beginning (or end) of the section?  The Official Python Tutorial has always assumed a certain programmer's competence.  It's up to them if they want to use the module or not.

> How about we simply add the warning from http://docs.python.org/3/library/pickle.html#module-pickle to the beginning (or end) of the section?

That is one possibility.

> The Official Python Tutorial has always assumed a certain programmer's competence.

Such a person is much more likely to know about and find use for 
language independent json than Python-specific pickle. The tutorial was 
written before json existed, but if it were written today, json would 
seem to me to be the better choice for first exposure to serialization. 
If someone cares enough to write a patch, I think it should be considered.

I won't question the usefulness of JSON.  I'm not a web programmer and have never used it.  From my interpretation of the tutorial, it seems that the section's purpose is for storing python objects.  If pickle is going to stay in the tutorial, I think a warning is imperative.  Either way, I think a section on JSON would be a welcome addition to the tutorial.

+1 adding a warning
+0 keeping pickle in the tutorial

Here's a patch that adds the warning, if we so choose to keep pickle in the tutorial.  It's taken verbatim from the pickle module's documentation.

The very fact that we would add a warning makes pickle unfit for the tutorial, IMHO. The tutorial should stick to simple enough stuff that doesn't need any warnings.

Here's a patch that focuses on the json module, with a smaller pickle section including the warning from the pickle docs.

Since this particular tutorial section was written long before the json module was part of the standard library, I think it makes sense to switch now we have the option.

pickle is definitely a useful tool, but now that JSON is available by default, it's now one to escalate to if JSON doesn't meet your needs, rather than the one to use by default.

Chris's patch looks generally good to me, although I think it could use a second paragraph in the pickle section. Perhaps something like:

"""JSON is much simpler to use safely than pickle, and offers broader interoperability with programs written in other languages. However, pickle has the advantage of supporting a much wider range of Python object types, including executable code, which means it can handle use cases that JSON can't (like sending code to another process for execution)"


This simpler/safer/less powerful API/technique vs more powerful/more dangerous API/technique trade-off is a fairly common one, so I think it's a good thing to have a concrete example of it in the tutorial rather than just dropping the reference to pickle entirely.

Sounds good to me.

Correction: you can't pickle executable code, you can pickle references to well-known objects (by name):

>>> def f(): pass
... 
>>> pickle.dumps(f)
b'\x80\x03c__main__\nf\nq\x00.'
>>> pickle.dumps(f.__code__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.PicklingError: Can't pickle <class 'code'>: attribute lookup code on builtins failed

Here is a modified patch. The changes are:
- pickle is deemphasized a lot more (it's moved into a "seealso")
- I replaced the terms "encoding" and "decoding" with "serializing" and "deserializing" (the former may be confusing for people who are already struggling to understand the bytes / unicode gap)
- I added a glossary entry for "text file" and pointed to it

Antoine's latest draft looks good to me.

I like Antoine's tutjson.patch.  commit it.  Thanks for noticing this Donald!

New changeset 90cf299dcf9b by Antoine Pitrou in branch '3.3':
Issue #18840: Introduce the json module in the tutorial, and deemphasize the pickle module.
http://hg.python.org/cpython/rev/90cf299dcf9b

New changeset 1009b77f59fd by Antoine Pitrou in branch 'default':
Issue #18840: Introduce the json module in the tutorial, and deemphasize the pickle module.
http://hg.python.org/cpython/rev/1009b77f59fd

New changeset 481b30bfe496 by Antoine Pitrou in branch '2.7':
Issue #18840: Introduce the json module in the tutorial, and deemphasize the pickle module.
http://hg.python.org/cpython/rev/481b30bfe496

Ok, I've committed it. Thanks!