Even though RFC 6265's cookie-value's cookie-octet definition says that whitespace characters are not allowed (see http://tools.ietf.org/html/rfc6265), there are non-compliant web sites that send cookie values with newlines.  cookielib.LWPCookieJar will save the bad value in a cookie file, but when it tries to load it, it probably is only splitting by newlines, so it crashes on the invalid line.  The easiest thing would probably be to just ignore cookies that have values with a newline.  (Note: I'm using Python 2.5, so this might have been fixed in a later version.)

Some notes: In 3.x, cookielib is http.cookiejar. LWPCookieJar is a subclass of FileCookieJar. The module follows RFC 2109 and RFC 2965. RFC 6265 is labelled a 'PROPOSED STANDARD' that 'obsoletes' 2965, but its status is unclear to me. RFC 2965 says it has already been obsoleted. Of course, what matters in practice is what sites do. Cookielib was written to work with obsolete 2109 because that is what was mostly used at the time.

Paulie, please check what the older standards, that are the basis of cookielib, say about newlines in cookies. Also, please load the current 2.7 and 3.2 and test with 'non-compliant' sites.

RFC 2109's Section 4.1 "Syntax: General" (http://www.ietf.org/rfc/rfc2109.txt) states that the attributes and values should be tokens, which the define as "(informally, a sequence of non-special, non-white space characters) from the HTTP/1.1 specification [RFC 2068]."

The one site that had sent me a cookie with a newline in a value is no longer doing that, but of course, there could be other sites with buggy code for sending cookies.

By "it crashes on the invalid line" do you mean Python raises an exception, prints a traceback, and exits? Or does it seqfault, dump core, or the Windows equivavlent?

Sorry, by "crash" I did mean that it raised an exception.  My program wasn't expecting cookielib to fail while reading a cookie file that it had written, so I didn't wrap the code to read the cookie file in a try..except.  I would imagine that most people wouldn't.  Do you agree?

It would be better to raise an exception* upon receiving a cookie. On the other hand, I presume cookies are stored in files that any process can mess with, so reading failures are always a possibility. So if you want to catch a (very rare) failure, to do something useful, then you should do so.

*My preference versus silent rejection. But I do not do web programming.

Unless the current doc says something I missed, I think either change from the current 'store anything' policy would be a feature request. This is assuming that current behavior is unchanged from 2.5.

OK, I'll wrap it in a try-except.  Do you think the documentation should updated to make users aware of this possible problem?

Thanks,
Paulie

This bug was reported in 2.5 and we don't have a test case attached in this report. Given many changes and age of the bug, I am closing this.

A specific report against the latest code will help us to fix this bug.