Add socket.create_server() utility function #80115

giampaolo · 2019-02-07T19:15:03Z

BPO	35934
Nosy	@loewis, @jaraco, @pitrou, @vstinner, @giampaolo, @asvetlov, @csabella, @pablogsal
PRs	bpo-35934: Add socket.create_server() utility function #11784 bpo-35934: Add socket.create_server() utility function #11784 bpo-35934: Add socket.create_server() utility function #11784

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-04-09.17:19:36.620>
created_at = <Date 2019-02-07.19:15:02.761>
labels = ['3.8', 'library']
title = 'Add socket.create_server() utility function'
updated_at = <Date 2019-04-09.17:32:30.921>
user = 'https://github.com/giampaolo'

bugs.python.org fields:

activity = <Date 2019-04-09.17:32:30.921>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-09.17:19:36.620>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2019-02-07.19:15:02.761>
creator = 'giampaolo.rodola'
dependencies = []
files = []
hgrepos = []
issue_num = 35934
keywords = ['patch', 'patch', 'patch']
message_count = 13.0
messages = ['335034', '335286', '335534', '335535', '339677', '339779', '339780', '339782', '339783', '339784', '339787', '339788', '339792']
nosy_count = 10.0
nosy_names = ['loewis', 'jaraco', 'pitrou', 'vstinner', 'giampaolo.rodola', 'josiah.carlson', 'asvetlov', 'neologix', 'cheryl.sabella', 'pablogsal']
pr_nums = ['11784', '11784', '11784']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35934'
versions = ['Python 3.8']

giampaolo · 2019-02-07T19:15:03Z

The main point of this patch is to automatize all the necessary tasks which are usually involved when creating a server socket, amongst which:

determining the right family based on address, similarly to socket.create_connection()
whether to use SO_REUSEADDR depending on the platform
set AI_PASSIVE flag for getaddrinfo()
set a more optimal default backlog

This is somewhat related to bpo-17561 which I prefer to leave pending for now (need to think about it more carefully). bpo-17561 is complementary to this one so it appears it can be integrated later (or never) without altering the base functionality implemented in here.

giampaolo · 2019-02-12T04:45:55Z

Closing this out as duplicate. Will continue in bpo-35934.

csabella · 2019-02-14T14:20:39Z

The previous comment is a self-reference. I believe the intention was to mark this as a duplicate of bpo-17561.

giampaolo · 2019-02-14T14:39:51Z

Correct. Sorry for the typo.

giampaolo · 2019-04-08T22:34:07Z

New changeset eb7e29f by Giampaolo Rodola in branch 'master':
bpo-35934: Add socket.create_server() utility function (GH-11784)
eb7e29f

vstinner · 2019-04-09T16:54:21Z

The change broke multiple buildbots, please fix buildbots or the change will be reverted:
#11784 (comment)

Revert on fail: https://pythondev.readthedocs.io/ci.html#revert-on-fail

vstinner · 2019-04-09T16:55:06Z

AMD64 Ubuntu Shared 3.x:

https://buildbot.python.org/all/#/builders/141/builds/1498

FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/_test_multiprocessing.py", line 4377, in test_wait_socket
    self.assertEqual(b''.join(v), expected)
AssertionError: b'1\n2\n3\n4\n5\n6\n7\n8\n9\n' != b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n'

vstinner · 2019-04-09T16:57:03Z

AMD64 FreeBSD CURRENT Shared 3.x:

https://buildbot.python.org/all/#/builders/168/builds/885

test_wait (test.test_multiprocessing_spawn.TestWait) ... ok
test_wait_integer (test.test_multiprocessing_spawn.TestWait) ... ok
test_wait_slow (test.test_multiprocessing_spawn.TestWait) ... ok
Process Process-62:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2397)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Process Process-60:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2398)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Process Process-59:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 302, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4337, in _child_test_wait_socket
    s.connect(address)
ConnectionResetError: [Errno 54] Connection reset by peer
/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/multiprocessing/process.py:318: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 2399)>
  traceback.print_exc()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Timeout (0:25:00)!
Thread 0x0000000800acc000 (most recent call first):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/socket.py", line 212 in accept
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/_test_multiprocessing.py", line 4361 in test_wait_socket
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/case.py", line 680 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/case.py", line 740 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/runner.py", line 176 in run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/support/__init__.py", line 1968 in _run_suite
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/support/__init__.py", line 2064 in run_unittest
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 178 in test_runner
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 182 in runtest_inner
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 137 in runtest
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 305 in rerun_failed_tests
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 623 in _main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 586 in main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/main.py", line 640 in main
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/__main__.py", line 2 in <module>
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/runpy.py", line 85 in _run_code
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/runpy.py", line 192 in _run_module_as_main
test_wait_socket (test.test_multiprocessing_spawn.TestWait) ... *** Error code 1

vstinner · 2019-04-09T16:58:40Z

s390x SLES 3.x:
https://buildbot.python.org/all/#/builders/16/builds/2625

FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
FAIL: test_wait_socket (test.test_multiprocessing_spawn.TestWait)
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_fork' in verbose mode
FAIL: test_wait_socket (test.test_multiprocessing_fork.TestWait)
Re-running test 'test_multiprocessing_spawn' in verbose mode

giampaolo · 2019-04-09T17:01:12Z

Those are old failures (+/- 17 hours ago). I committed:
#12735
...which fixed them.

vstinner · 2019-04-09T17:19:36Z

Those are old failures (+/- 17 hours ago). I committed:
#12735

Oh, you wrote "BPO-" rather than "bpo-" and so your commit wasn't mentioned here:

commit 8702b67
Author: Giampaolo Rodola <g.rodola@gmail.com>
Date: Tue Apr 9 04:42:06 2019 +0200

BPO-17561: set create_server backlog default to None (GH-12735)

It turns out doing socket.listen(0) does not equal to "choose a
reasonable default". It actually means "set backlog to 0".
As such set backlog=None as the default for socket.create_server.
Fixes the following BB failures:
https://github.com/python/cpython/pull/11784#issuecomment-481036369
Ref. BPO-1756, python/cpython#55993.

...which fixed them.

Oh, great! Thanks for the fix.

Sorry, I was 10 days late to handle emails from the buildbot-status mailing list. I only reported failures, I didn't check if it has been fixed in the meanwhile.

I close again the issue.

giampaolo · 2019-04-09T17:21:33Z

No problem. My bad that I mistyped the "BPO" thing.

vstinner · 2019-04-09T17:32:31Z

I was sure that I already reported the issue... In fact, I commented bpo-17561 :-)

giampaolo added 3.8 only security fixes stdlib Python modules in the Lib dir labels Feb 7, 2019

giampaolo closed this as completed Feb 12, 2019

vstinner changed the title ~~Add socket.bind_socket() utility function~~ Add socket.create_server() utility function Feb 14, 2019

vstinner reopened this Apr 9, 2019

vstinner closed this as completed Apr 9, 2019

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add socket.create_server() utility function #80115

Add socket.create_server() utility function #80115

giampaolo commented Feb 7, 2019

giampaolo commented Feb 7, 2019

giampaolo commented Feb 12, 2019

csabella commented Feb 14, 2019

giampaolo commented Feb 14, 2019

giampaolo commented Apr 8, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

giampaolo commented Apr 9, 2019

vstinner commented Apr 9, 2019

giampaolo commented Apr 9, 2019

vstinner commented Apr 9, 2019

Add socket.create_server() utility function #80115

Add socket.create_server() utility function #80115

Comments

giampaolo commented Feb 7, 2019

giampaolo commented Feb 7, 2019

giampaolo commented Feb 12, 2019

csabella commented Feb 14, 2019

giampaolo commented Feb 14, 2019

giampaolo commented Apr 8, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

vstinner commented Apr 9, 2019

giampaolo commented Apr 9, 2019

vstinner commented Apr 9, 2019

giampaolo commented Apr 9, 2019

vstinner commented Apr 9, 2019