classification
Title: MozillaCookieJar ignores HttpOnly cookies
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: Jacob Taylor, dlenski, douyuan, jdetrey, jjlee, loewis, mt0321, orsenthil, python-dev, terry.reedy
Priority: normal Keywords: patch

Created on 2008-02-25 16:39 by douyuan, last changed 2020-11-18 05:27 by orsenthil. This issue is now closed.

Files
File name Uploaded Description Edit
_MozillaCookieJar.diff douyuan, 2008-02-25 16:39 a quick & dirty fix
httponly.patch jdetrey, 2015-01-07 09:57 patch review
Pull Requests
URL Status Linked Edit
PR 22798 closed python-dev, 2020-10-20 00:25
PR 17471 Jacob Taylor, 2020-11-15 20:42
Messages (17)
msg62985 - (view) Author: Dou Yuan (douyuan) Date: 2008-02-25 16:39
HttpOnly cookie in Firefox's cookies.txt begins with "#HttpOnly_" now,
just like a comment, e.g.:

#HttpOnly_.rad.live.com    TRUE    /    FALSE    1258200001    FC09    FB=
#HttpOnly_service.ilib.cn    FALSE    /    FALSE    1209905939   
.ASPXANONYMOUS   
JMeD5-atyAEkAAAAYjZlNDUyNDAtOGQ4ZC00NTEyLTljN2EtMzNkODM3M2JjMjFivtX6ikB7Iv0jRJBJs9ftggv_a2k

Since no obvious need, there are no patches for save method and
cookielib.Cookie class.
msg74822 - (view) Author: John J Lee (jjlee) Date: 2008-10-15 21:42
I think firefox 3 no longer writes cookies.txt (it writes cookies.sqlite
instead).

Can anybody point out a version of firefox that wrote this HttpOnly
information to cookies.txt, so the patch can be tested?
msg109819 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-07-10 05:55
MozillaCookieJar is now a class in http.cookiejar, so patch would need update. Is this still used enough to bother?
msg109958 - (view) Author: Dou Yuan (douyuan) Date: 2010-07-11 03:42
Firefox no longer use cookies.txt. I think this patch is useless.
msg110058 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-07-12 00:04
Would you suggest removing MozillaCookieJar from the module?
(Through the normal warn-deprecate-remove process.)
msg110121 - (view) Author: John J Lee (jjlee) Date: 2010-07-12 18:18
Is deprecation really necessary?  lynx still uses that format.  lynx doesn't write the header that MozillaCookieJar insists on being present, but a trivial subclass can read cookies files written by lynx.
msg233571 - (view) Author: Jérémie Detrey (jdetrey) * Date: 2015-01-07 09:57
Dear all,

In fact, this cookie.txt format is still used by curl. For instance, see

  https://github.com/bagder/curl/blob/curl-7_39_0/lib/cookie.c#L644

which clearly shows support for the "#HttpOnly_" prefix. Therefore, supporting this format in http.cookiejar.MozillaCookieJar seems quite relevant to me.

Attached is an updated patch.

Kind regards,
Jérémie.
msg300920 - (view) Author: Mike Thomas (mt0321) Date: 2017-08-27 17:50
Can this issue be reopened? As Jérémie stated, curl uses this format and outputs cookie files using the #HttpOnly_ prefix. I also found at least one project that is working around lack of this support:
https://code.google.com/archive/p/git-repo/
https://gerrit.googlesource.com/git-repo/+/master/subcmds/sync.py#995
      # Python doesn't understand cookies with the #HttpOnly_ prefix
      # Since we're only using them for HTTP, copy the file temporarily,
      # stripping those prefixes away.

One potential improvement for the proposed patch: instead of just stripping out #HttpOnly_, this attribute should be set on the Cookie that is created, within the 'rest' dict (rest={'HttpOnly': True}). The Morsel class is already aware of this attribute, as is the 'requests.cookies' module.
msg367468 - (view) Author: Daniel Lenski (dlenski) * Date: 2020-04-27 22:19
Also confused about why this was closed.

This format is still frequently used. In the absence of a solution in the standard library, I'm using this kludge to strip the leading `#HttpOnly_`.


    from tempfile import NamedTemporaryFile
    from http.cookiejar import MozillaCookieJar
    from contextlib import contextmanager

    def fix_cookie_jar_file(orig_cookiejarfile):
        with NamedTemporaryFile(mode='w+') as cjf:
            with open(orig_cookiejarfile, 'r') as ocf:
            for l in ocf:            
                cjf.write(l[10:] if l.startswith('#HttpOnly_') else l)
            cjf.seek(0)
        yield cjf.name

    MozillaCookieJar(filename=fix_cookie_jar_file(orig_cookiejarfile))
msg367469 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-04-27 23:52
This issue was closed as useless for Firefox in 2010 by the original poster, msg109958.  My participation here is only as tracker triager, as I only have a consumer knowledge of cookies.  Unfortunately, there is no core developer expert for http, let alone the http.cookiejar.  The person who once handled some cookie related patches is no longer active.

Adding a patch to a closed issue is somewhat useless.  In any case, a possible revised PR would be needed.  My suggestion is to ask on python-ideas whether this enhancement might be accepted now and whether better to reopen this issue or open a new one.
msg379003 - (view) Author: Daniel Lenski (dlenski) * Date: 2020-10-19 19:54
I've got a patch that will address both loading and saving of "HTTP-only" cookies: https://github.com/python/cpython/compare/master...dlenski:patch-1

Testing/feedback before I submit as a PR would be very welcome.
msg379312 - (view) Author: Daniel Lenski (dlenski) * Date: 2020-10-22 16:27
@terry.reedy, it looks like my PR just needs a core developer to review it. Would you mind taking a look? :-)

https://github.com/python/cpython/pull/22798
msg379398 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-23 02:49
https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies
Describes the purpose of the HttpOnly attribute used in PR.
msg381034 - (view) Author: Daniel Lenski (dlenski) * Date: 2020-11-15 20:39
Issue #38976 is a duplicate of this one, and now closed by https://github.com/python/cpython/pull/17471
msg381039 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-11-15 22:40
So, is anything more needed, or should PR-22798 and this issue be closed?
msg381312 - (view) Author: Daniel Lenski (dlenski) * Date: 2020-11-18 05:23
This can be closed.
msg381313 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2020-11-18 05:27
Yes. The required 'feature' was introduced through https://github.com/python/cpython/pull/17471/ even as the patches were slightly different. But keeping https://github.com/python/cpython/pull/17471/ seems fine and we can close this ticket and the PR.
History
Date User Action Args
2020-11-18 05:27:37orsenthilsetstatus: open -> closed
resolution: duplicate
messages: + msg381313

stage: patch review -> resolved
2020-11-18 05:23:38dlenskisetmessages: + msg381312
2020-11-15 22:40:18terry.reedysetnosy: + orsenthil
messages: + msg381039
2020-11-15 20:42:44Jacob Taylorsetnosy: + Jacob Taylor

pull_requests: + pull_request22197
stage: patch review
2020-11-15 20:39:03dlenskisetmessages: + msg381034
2020-10-23 02:49:11terry.reedysetmessages: + msg379398
2020-10-23 02:30:27terry.reedysetstatus: closed -> open
assignee: loewis ->
versions: - Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
2020-10-22 16:27:41dlenskisetmessages: + msg379312
2020-10-20 00:25:54python-devsetnosy: + python-dev

pull_requests: + pull_request21754
2020-10-20 00:17:25dlenskisettitle: MozillaCookieJar ignore HttpOnly cookies -> MozillaCookieJar ignores HttpOnly cookies
2020-10-19 19:54:57dlenskisetmessages: + msg379003
versions: + Python 3.7, Python 3.8, Python 3.9, Python 3.10
2020-04-27 23:52:04terry.reedysetmessages: + msg367469
2020-04-27 22:19:23dlenskisetnosy: + dlenski
messages: + msg367468
2017-08-27 17:50:28mt0321setnosy: + mt0321
messages: + msg300920
2015-01-07 10:04:01jdetreysetversions: + Python 3.3, Python 3.4, Python 3.5, Python 3.6
2015-01-07 09:57:59jdetreysetfiles: + httponly.patch
nosy: + jdetrey
messages: + msg233571

2010-07-12 18:18:09jjleesetmessages: + msg110121
2010-07-12 00:04:46terry.reedysetmessages: + msg110058
2010-07-11 03:42:44douyuansetstatus: open -> closed

messages: + msg109958
2010-07-10 05:55:33terry.reedysetnosy: + terry.reedy

messages: + msg109819
versions: + Python 3.2, - Python 2.6
2008-10-15 21:42:35jjleesetmessages: + msg74822
2008-10-09 18:56:00jjleesetnosy: + jjlee
2008-03-20 03:23:55jafosetpriority: normal
assignee: loewis
type: enhancement
nosy: + loewis
2008-02-25 16:39:05douyuancreate