This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: cookielib.LWPCookieJar breaks on cookie values with a newline
Type: behavior Stage: resolved
Components: Library (Lib) Versions:
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: orsenthil, paulie4, terry.reedy
Priority: normal Keywords:

Created on 2011-09-12 19:14 by paulie4, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (8)
msg143928 - (view) Author: Paulie Pena (paulie4) Date: 2011-09-12 19:14
Even though RFC 6265's cookie-value's cookie-octet definition says that whitespace characters are not allowed (see, there are non-compliant web sites that send cookie values with newlines.  cookielib.LWPCookieJar will save the bad value in a cookie file, but when it tries to load it, it probably is only splitting by newlines, so it crashes on the invalid line.  The easiest thing would probably be to just ignore cookies that have values with a newline.  (Note: I'm using Python 2.5, so this might have been fixed in a later version.)
msg144149 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-09-16 18:58
Some notes: In 3.x, cookielib is http.cookiejar. LWPCookieJar is a subclass of FileCookieJar. The module follows RFC 2109 and RFC 2965. RFC 6265 is labelled a 'PROPOSED STANDARD' that 'obsoletes' 2965, but its status is unclear to me. RFC 2965 says it has already been obsoleted. Of course, what matters in practice is what sites do. Cookielib was written to work with obsolete 2109 because that is what was mostly used at the time.

Paulie, please check what the older standards, that are the basis of cookielib, say about newlines in cookies. Also, please load the current 2.7 and 3.2 and test with 'non-compliant' sites.
msg144538 - (view) Author: Paulie Pena (paulie4) Date: 2011-09-26 13:40
RFC 2109's Section 4.1 "Syntax: General" ( states that the attributes and values should be tokens, which the define as "(informally, a sequence of non-special, non-white space characters) from the HTTP/1.1 specification [RFC 2068]."

The one site that had sent me a cookie with a newline in a value is no longer doing that, but of course, there could be other sites with buggy code for sending cookies.
msg144547 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-09-26 19:16
By "it crashes on the invalid line" do you mean Python raises an exception, prints a traceback, and exits? Or does it seqfault, dump core, or the Windows equivavlent?
msg144548 - (view) Author: Paulie Pena (paulie4) Date: 2011-09-26 19:45
Sorry, by "crash" I did mean that it raised an exception.  My program wasn't expecting cookielib to fail while reading a cookie file that it had written, so I didn't wrap the code to read the cookie file in a try..except.  I would imagine that most people wouldn't.  Do you agree?
msg144549 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-09-26 20:40
It would be better to raise an exception* upon receiving a cookie. On the other hand, I presume cookies are stored in files that any process can mess with, so reading failures are always a possibility. So if you want to catch a (very rare) failure, to do something useful, then you should do so.

*My preference versus silent rejection. But I do not do web programming.

Unless the current doc says something I missed, I think either change from the current 'store anything' policy would be a feature request. This is assuming that current behavior is unchanged from 2.5.
msg144550 - (view) Author: Paulie Pena (paulie4) Date: 2011-09-26 21:13
OK, I'll wrap it in a try-except.  Do you think the documentation should updated to make users aware of this possible problem?

msg392014 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2021-04-27 01:33
This bug was reported in 2.5 and we don't have a test case attached in this report. Given many changes and age of the bug, I am closing this.

A specific report against the latest code will help us to fix this bug.
Date User Action Args
2022-04-11 14:57:21adminsetgithub: 57175
2021-04-27 01:33:23orsenthilsetstatus: open -> closed

messages: + msg392014
stage: resolved
2011-09-26 21:13:06paulie4setmessages: + msg144550
2011-09-26 20:40:12terry.reedysetmessages: + msg144549
2011-09-26 19:45:34paulie4setmessages: + msg144548
2011-09-26 19:16:48terry.reedysetmessages: + msg144547
2011-09-26 13:40:33paulie4setmessages: + msg144538
2011-09-26 13:16:05orsenthilsetassignee: orsenthil

nosy: + orsenthil
2011-09-16 18:58:40terry.reedysetnosy: + terry.reedy
messages: + msg144149
2011-09-12 19:14:40paulie4create