This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: tests fail due to unsupported SO_REUSEPORT when building Python 3.3.2-r2
Type: behavior Stage: resolved
Components: Build Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: RubyTuesdayDONO, dmalcolm, gregory.p.smith
Priority: normal Keywords: easy

Created on 2013-12-05 20:59 by RubyTuesdayDONO, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (7)
msg205320 - (view) Author: Reuben Garrett (RubyTuesdayDONO) Date: 2013-12-05 20:59
When building Python 3.3.2-r2 from Gentoo's Portage tree [1], I encountered two failed tests which probably should not have been attempted on my OS (Gentoo 3.7.10): test_bind_port and test_find_unused_port both use the SO_REUSEPORT socket option which, to the best of my knowledge is only available since kernel version 3.9. I was able to build by skipping the tests — but I believe the tests are there for a reason, and it would be best if a test that is known to fail should be skipped (or replaced with a fallback that is likely to succeed). Issue # 16594 [3] may be related (not sure).

Is it possible to detect the kernel version and skip (or modify) these tests if SO_REUSEPORT is not available? Better yet (since even a 3.9 kernel could have it disabled) — try the test with SO_REUSEPORT, but trap the exception for lack of OS support, print a warning, and try again without SO_REUSEPORT. 

+=== excerpt of portage build log:
======================================================================
ERROR: test_bind_port (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.Lib/test/test_support.py", line 87, in test_bind_port
    support.bind_port(s)
  File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.Lib/test/support.py", line 548, in bind_port
    if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) == 1:
OSError: [Errno 92] Protocol not available

======================================================================
ERROR: test_find_unused_port (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.Lib/test/test_support.py", line 80, in test_find_unused_port
    port = support.find_unused_port()
  File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.Lib/test/support.py", line 522, in find_unused_port
    port = bind_port(tempsock)
  File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.Lib/test/support.py", line 548, in bind_port
    if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) == 1:
OSError: [Errno 92] Protocol not available
===+

[1]: https://packages.gentoo.org/package/dev-lang/python
[2]: https://lwn.net/Articles/542629/
[3]: http://bugs.python.org/issue16594
msg205343 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2013-12-06 01:41
[FWIW, this looks similar to an issue I ran into on Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=913732
which was due to a mismatch between the kernel headers on the system vs the actually running kernel.  I patched around it there with a downstream-only patch (our builds are done on chroots on RHEL-5 boxes, so such mismatches tend to occur)]
msg205479 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2013-12-07 19:35
I fixed this in 3.3 and 3.4 several weeks ago.

http://hg.python.org/cpython/rev/9791c5d55f52
http://hg.python.org/cpython/rev/00766fa3366b
msg206073 - (view) Author: Reuben Garrett (RubyTuesdayDONO) Date: 2013-12-13 13:15
On Sat, Dec 7, 2013 at 1:35 PM, Gregory P. Smith <report@bugs.python.org>wrote:

> I fixed this in 3.3 and 3.4 several weeks ago.
>
msg206074 - (view) Author: Reuben Garrett (RubyTuesdayDONO) Date: 2013-12-13 13:19
On Sat, Dec 7, 2013 at 1:35 PM, Gregory P. Smith <report@bugs.python.org>
wrote:
> I fixed this in 3.3 and 3.4 several weeks ago.

Sorry for the duplicate message a few moments ago … "fat-fingered" my send
button.

So, has the fix just not reached the Portage tree yet? I don't fully
understand how upstream commits reach Portage, and would appreciate any
direction you might have to avoid reporting something already fixed.
msg206202 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2013-12-14 20:38
You didn't do anything wrong. You were running 3.3.3 which is the latest release of 3.3.  Portage is Gentoo's thing, you'd have to ask the gentoo python portage ebuild maintainer and point them at the commit with the additional patch to apply if you want it fixed there.  (or file a bug on gentoo's bug tracking system and point it at this one)

In general gentoo is one of the more up to date distros (compared to everything else) but it is always helpful to try checking out and compiling the latest source tree from hg.python.org as described in http://docs.python.org/devguide/ (on the 3.3 branch in your case) when reporting an issue.

I just happened to have noticed this problem myself and fixed it without bothering to file an issue about it a few weeks before you ran into it. :)
msg206302 - (view) Author: Reuben Garrett (RubyTuesdayDONO) Date: 2013-12-16 13:08
On Sat, Dec 14, 2013 at 2:38 PM, Gregory P. Smith <report@bugs.python.org>
wrote:
> ask the gentoo python portage ebuild maintainer and point them at the
commit with the additional patch to apply if you want it fixed there.  (or
file a bug on gentoo's bug tracking system and point it at this one)
> …
> try checking out and compiling the latest source tree from hg.python.orgas described in
http://docs.python.org/devguide/ (on the 3.3 branch in your case) when
reporting an issue.

That's actually really good advice (applicable to other projects beyond
Python) — thank you so much, Gregory!
History
Date User Action Args
2022-04-11 14:57:55adminsetgithub: 64100
2013-12-16 13:08:52RubyTuesdayDONOsetmessages: + msg206302
2013-12-14 20:38:03gregory.p.smithsetmessages: + msg206202
2013-12-13 13:19:18RubyTuesdayDONOsetmessages: + msg206074
2013-12-13 13:15:36RubyTuesdayDONOsetmessages: + msg206073
2013-12-07 19:35:25gregory.p.smithsetstatus: open -> closed

type: behavior
assignee: gregory.p.smith
versions: + Python 3.4
nosy: + gregory.p.smith

messages: + msg205479
resolution: fixed
stage: resolved
2013-12-06 01:41:51dmalcolmsetnosy: + dmalcolm
messages: + msg205343
2013-12-05 21:09:58gvanrossumsetkeywords: + easy
2013-12-05 20:59:45RubyTuesdayDONOcreate