This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: socket: change listen() default backlog from 128 to 4096?
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: ammar2, giampaolo.rodola, njs, vstinner, yselivanov
Priority: normal Keywords:

Created on 2019-11-05 14:45 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (9)
msg356037 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-05 14:45
Currently, socket.socket.listen() called with no argument limits the default backlog to 128:

    /* We try to choose a default backlog high enough to avoid connection drops
     * for common workloads, yet not too high to limit resource usage. */
    int backlog = Py_MIN(SOMAXCONN, 128);

I just saw an article suggesting to use 4096 instead:
https://blog.cloudflare.com/syn-packet-handling-in-the-wild/

On my Fedora 30, listen() manual page says:

       The  behavior of the backlog argument on TCP sockets changed with Linux
       2.2.  Now it specifies the  queue  length  for  completely  established
       sockets  waiting  to  be  accepted, instead of the number of incomplete
       connection requests.  The maximum length of the  queue  for  incomplete
       sockets  can be set using /proc/sys/net/ipv4/tcp_max_syn_backlog.  When
       syncookies are enabled there is no logical maximum length and this set‐
       ting is ignored.  See tcp(7) for more information.

       If    the   backlog   argument   is   greater   than   the   value   in
       /proc/sys/net/core/somaxconn, then it is  silently  truncated  to  that
       value;  the  default  value  in  this  file  is 128.  In kernels before
       2.4.25, this limit was a hard coded value, SOMAXCONN,  with  the  value
       128.

Python 3.8 new socket.create_server() calls sock.listen() by default: so use Py_MIN(SOMAXCONN, 128) by default.
msg356038 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-05 14:47
> I just saw an article suggesting to use 4096 instead:

In fact, it's a change in the Linux kernel directly proposed by Eric Dumazet (Google) for Linux kernel 5.5:
https://lore.kernel.org/netdev/20191030163620.140387-1-edumazet@google.com/

The change has already been merged upstream:

/* Maximum queue length specifiable by listen.  */
#define SOMAXCONN	4096

https://github.com/torvalds/linux/blob/a99d8080aaf358d5d23581244e5da23b35e340b9/include/linux/socket.h#L265
msg356049 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2019-11-05 18:04
Just for some more reference points from "production" python web servers:

* gunicorn - 2048 (https://github.com/benoitc/gunicorn/blob/678b326dc030b450717ec505df69863dcd6fb716/docs/source/settings.rst#backlog)

* tornado - 128 (https://github.com/tornadoweb/tornado/blob/c50aed0f96d92f9b0ef4cd0837c0104f140ca77e/tornado/tcpserver.py#L178)

* uwsgi - 100 (https://github.com/unbit/uwsgi/blob/3149df02ed443131c54ea6afb29fcbb0ed4d1139/core/init.c#L115)
msg357956 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2019-12-06 23:41
Trio uses 65535 as the default backlog argument. There's a big comment here explaining why: https://github.com/python-trio/trio/blob/master/trio/_highlevel_open_tcp_listeners.py

This doesn't actually set the backlog to 65635; instead it tells the kernel to use the biggest backlog that it feels comfortable with (e.g. /proc/sys/net/core/somaxconn on Linux).
msg357957 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2019-12-06 23:49
Another subtlety that that code handles, and that the stdlib socket module might also want to handle: if the user passes in a custom backlog argument that's >65535, then we silently replace it with 66535 before passing it to the system. The reason is that many systems will silently truncate this value to 16 bits, so if a user explicitly passes in 65536, what they get is a backlog of 1, which is probably not what they were hoping for.
msg358092 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-12-09 14:11
> https://github.com/python-trio/trio/blob/master/trio/_highlevel_open_tcp_listeners.py

Extract:

# A large backlog can also use a bit more kernel memory, but this seems fairly
# negligible these days.

That's the main question here. Python is used on various platforms for various use cases.

Extract of the current C code:

    /* We try to choose a default backlog high enough to avoid connection drops
     * for common workloads, yet not too high to limit resource usage. */
    int backlog = Py_MIN(SOMAXCONN, 128);

Maybe the status quo is just fine and this issue should be closed. It's trivial to override the default. This issue is stricted to the *default* value.
msg358093 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-12-09 14:12
gunicorn, tornado, uwsgi, trio are specialized use cases which expect high concurrency.

Maybe a tradeoff may be to document the default and suggest different values depending on the use cases, and/or to point to documentation.
msg358143 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2019-12-09 21:28
Yeah, I don't think it matters that much. There are lots of random gotchas that you have to watch out for using the socket module.

To be clear: I do still think that using a large value by default, and clamping user input values at 65535, would be improvements. I just don't think they're important enough to spend energy arguing about it :-).

The reason I'm not worried about large backlogs wasting kernel memory is that servers all call accept as fast as they can. This means any incoming connection flood will end up taking memory regardless of the size of the kernel-side buffer. And I'm pretty sure the kernel buffer ia a linked list, so setting it to a large value doesn't cost anything if you're not using it.

Anyway asyncio at least should probably update its default.
msg361562 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-02-07 10:35
There is no clear consensus to change socket.listen() default backlog, so I close the issue.

> Anyway asyncio at least should probably update its default.

If someone cares, please open a separated issue ;-)

> To be clear: I do still think that using a large value by default, and clamping user input values at 65535, would be improvements. I just don't think they're important enough to spend energy arguing about it :-).

I dislike having to workaround operating system bugs. If the function misbehaves, I would prefer to see the OS being fixed, than Python trying to guess which value is supposed to work or not.

       int listen(int sockfd, int backlog);

backlog type is an int: its maximum value is INT_MAX. If the kernel rejects values larger than 65535, the kernel and/or the libc should reject such value.
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82880
2020-02-07 10:35:05vstinnersetstatus: open -> closed
resolution: rejected
messages: + msg361562

stage: resolved
2019-12-09 21:28:39njssetmessages: + msg358143
2019-12-09 14:12:31vstinnersetmessages: + msg358093
2019-12-09 14:11:36vstinnersetmessages: + msg358092
2019-12-06 23:49:42njssetmessages: + msg357957
2019-12-06 23:41:43njssetmessages: + msg357956
2019-12-06 22:56:38vstinnersetnosy: + njs, yselivanov
2019-11-05 18:04:07ammar2setnosy: + ammar2
messages: + msg356049
2019-11-05 14:47:51vstinnersetmessages: + msg356038
2019-11-05 14:45:47vstinnercreate