classification
Title: getaddrinfo is wrongly considered thread safe on linux
Type: crash Stage: resolved
Components: Extension Modules Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: Julien.Palard, gregory.p.smith, haypo
Priority: normal Keywords:

Created on 2014-04-14 17:21 by Julien.Palard, last changed 2014-06-26 04:06 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
test_getaddrinfo.c haypo, 2014-04-16 07:58
Messages (7)
msg216121 - (view) Author: Julien Palard (Julien.Palard) Date: 2014-04-14 17:21
I just found that python consider linux implementation of getaddrinfo thread safe :

./python2.6-2.6.8/Modules/socketmodule.c:180

/* On systems on which getaddrinfo() is believed to not be thread-safe,                                                                                                                                                             
   (this includes the getaddrinfo emulation) protect access with a lock. */
#if defined(WITH_THREAD) && (defined(__APPLE__) || \
    (defined(__FreeBSD__) && __FreeBSD_version+0 < 503000) || \
    defined(__OpenBSD__) || defined(__NetBSD__) || \
    defined(__VMS) || !defined(HAVE_GETADDRINFO))
#define USE_GETADDRINFO_LOCK
#endif

Badly, it's wrong on my version of linux, and maybe others : https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=722075

So for me, looping in three threads on getaddrinfo leads me to a rapid deadlock, while reading on a NETLINK socket, after another thread wrote a DNS query on it and trying to get a response on it.

It may only be reproductible when your getaddrinfo use a NETLINK to get informations about your interfaces before doing the DNS query.
msg216168 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-04-14 18:47
Can you attach some python code that reproduces this for you?  According to both of the references below it doesn't sound like this is supposed to be a problem.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/freeaddrinfo.html claims "The freeaddrinfo() and getaddrinfo() functions shall be thread-safe."

Linux man pages claim "getaddrinfo() is reentrant".
msg216447 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-16 07:51
> It may only be reproductible when your getaddrinfo use a NETLINK to get informations about your interfaces before doing the DNS query.

What is your operation system? Name and version. What is your version of the C library? What is your Python version? Can you provide an example to reproduce the issue?
msg216449 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-16 07:58
test_getaddrinfo.c: C program to run getaddrinfo() concurrently in different threads, it comes from the Debian issue.

I ran this program with 10 threads, I stopped it after between 3000 and 5000 tries (depending on the thread). I'm running Fedora 20: Linux kernel 3.13.9-200.fc20.x86_64 and glibc 2.18.
msg216450 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-16 07:59
Can you provide the C and Python backtrace of all threads of your program?
msg217231 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-27 00:23
@Julien.Palard: Ping? Without more information, I would suggest to close the issue.
msg221590 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-06-26 04:06
The upstream Debian issue contains a fix for the bug in eglibc. Python should not attempt to work around this. Distros need to fix their libc if they shipped a broken one.
History
Date User Action Args
2014-06-26 04:06:00gregory.p.smithsetstatus: open -> closed
type: crash
messages: + msg221590

components: + Extension Modules
resolution: not a bug
stage: resolved
2014-04-27 00:23:54hayposetmessages: + msg217231
2014-04-16 07:59:12hayposetmessages: + msg216450
2014-04-16 07:58:50hayposetfiles: + test_getaddrinfo.c

messages: + msg216449
2014-04-16 07:51:27hayposetmessages: + msg216447
2014-04-16 07:48:06hayposetnosy: + haypo
2014-04-14 18:47:14gregory.p.smithsetmessages: + msg216168
2014-04-14 18:36:05gregory.p.smithsetassignee: gregory.p.smith

nosy: + gregory.p.smith
versions: + Python 2.7, Python 3.4, Python 3.5
2014-04-14 17:21:43Julien.Palardcreate