classification
Title: Memory leak in SSLSocket.getpeercert()
Type: Stage: resolved
Components: Versions: Python 3.6, Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alex, benjamin.peterson, christian.heimes, dstufft, giampaolo.rodola, janssen, martin.panter, matrixise, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2015-11-06 20:08 by alex, last changed 2015-11-14 13:54 by alex. This issue is now closed.

Files
File name Uploaded Description Edit
issue25569.patch matrixise, 2015-11-06 20:47 review
issue25569-2.patch matrixise, 2015-11-07 22:01 review
Messages (12)
msg254210 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2015-11-06 20:08
Run the following code:

import socket
import ssl
import sys


def main():
    ctx = ssl.create_default_context()
    s = socket.create_connection(('www.bing.com', 443))
    s = ctx.wrap_socket(s, server_hostname='www.bing.com')
    while True:
        s.getpeercert()

        sys.stderr.write('.')
        sys.stderr.flush()


if __name__ == '__main__':
    main()



and observe memory usage, you'll see it increases by a MB/s or so.
msg254211 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2015-11-06 20:09
A probably source of the leak is here: https://github.com/python/cpython/blob/master/Modules/_ssl.c#L1073-L1076 `dps` is never freed. (This is with OpenSSL 0.9.8zg)
msg254212 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-06 20:30
Hi Alex,

not sure, but I think the right function to free the memory is AUTHORITY_INFO_ACCESS_free(dps)

I will provide a patch for a review asap
msg254213 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2015-11-06 20:38
I think you want sk_DIST_POINT_free actually.
msg254215 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-06 20:47
Maybe you are right, I didn't see that this function returns a pointer in function of the parameter. sorry.


Could you try with this patch, really sorry but my internet connection is really slow and I have a problem with the _ssl module on my laptop (problem with the compilation and the dependencies because I am on OSX).

Thank you
msg254251 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2015-11-07 01:50
Tests pass and the original script runs without a leak using this patch. It could probably be shorter if we converted from local returns to `goto fail` or something, but I don't really have an opinion.
msg254271 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-07 14:35
Hi alex,

Thank you for your feeback, I will propose an other patch with your remarks.

Stephane
msg254301 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-07 22:01
sorry for the delay.

here is a new version of my patch, if I can have feedback about this patch, I will really appreciate.

thank you
msg254519 - (view) Author: Roundup Robot (python-dev) Date: 2015-11-12 06:14
New changeset 10c3646b2d59 by Benjamin Peterson in branch '2.7':
fix memory leak in _get_crl_dp (closes #25569)
https://hg.python.org/cpython/rev/10c3646b2d59

New changeset aabe273b20ab by Benjamin Peterson in branch '3.4':
fix memory leak in _get_crl_dp (closes #25569)
https://hg.python.org/cpython/rev/aabe273b20ab

New changeset 07a298572d93 by Benjamin Peterson in branch '3.5':
merge 3.5 (#25569)
https://hg.python.org/cpython/rev/07a298572d93

New changeset fb55b1ab43fc by Benjamin Peterson in branch 'default':
merge 3.5 (#25569)
https://hg.python.org/cpython/rev/fb55b1ab43fc
msg254625 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-13 21:55
Seems to have broken the SSL module. Looks like you typoed “dps” as “dsp” in some conditional code.

http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.4/builds/1326/steps/compile/logs/stdio

building '_ssl' extension
gcc -fno-strict-aliasing -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Include -I/Users/buildbot/buildarea/3.4.murray-snowleopard/build -c /Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c -o build/temp.macosx-10.6-x86_64-3.4-pydebug/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.o
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c: In function ‘_get_crl_dp’:
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c:1084: error: ‘dsp’ undeclared (first use in this function)
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c:1084: error: (Each undeclared identifier is reported only once
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c:1084: error: for each function it appears in.)
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c: In function ‘_setup_ssl_threads’:
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Modules/_ssl.c:3828: warning: comparison is always false due to limited range of data type
msg254644 - (view) Author: Roundup Robot (python-dev) Date: 2015-11-14 08:11
New changeset 3b9fb8ebf44f by Benjamin Peterson in branch '2.7':
fix build with older openssl (#25569)
https://hg.python.org/cpython/rev/3b9fb8ebf44f

New changeset f13a75544b6f by Benjamin Peterson in branch '3.4':
fix build with older openssl (#25569)
https://hg.python.org/cpython/rev/f13a75544b6f

New changeset 83ea7e75605a by Benjamin Peterson in branch '3.5':
merge 3.4 (#25569)
https://hg.python.org/cpython/rev/83ea7e75605a

New changeset 8ac09e46ca45 by Benjamin Peterson in branch 'default':
merge 3.5 (#25569)
https://hg.python.org/cpython/rev/8ac09e46ca45
msg254660 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2015-11-14 13:54
fixed
History
Date User Action Args
2015-11-14 13:54:21alexsetstatus: open -> closed

messages: + msg254660
2015-11-14 08:11:29python-devsetmessages: + msg254644
2015-11-13 21:56:14martin.pantersetnosy: + benjamin.peterson
2015-11-13 21:55:13martin.pantersetstatus: closed -> open
versions: + Python 3.4
nosy: + martin.panter

messages: + msg254625
2015-11-12 06:14:51python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg254519

resolution: fixed
stage: resolved
2015-11-07 22:01:52matrixisesetfiles: + issue25569-2.patch

messages: + msg254301
2015-11-07 14:35:42matrixisesetmessages: + msg254271
2015-11-07 01:50:53alexsetmessages: + msg254251
2015-11-06 20:47:36matrixisesetfiles: + issue25569.patch
keywords: + patch
messages: + msg254215
2015-11-06 20:38:03alexsetmessages: + msg254213
2015-11-06 20:30:33matrixisesetnosy: + matrixise
messages: + msg254212
2015-11-06 20:09:54alexsetmessages: + msg254211
2015-11-06 20:08:08alexsetnosy: + janssen, pitrou, giampaolo.rodola, christian.heimes, dstufft
2015-11-06 20:08:01alexcreate