classification
Title: Add the SIO_LOOPBACK_FAST_PATH option to socket.ioctl
Type: enhancement Stage: resolved
Components: Extension Modules, Windows Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: Daniel Stokes, berker.peksag, martin.panter, paul.moore, pitrou, python-dev, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2016-03-11 03:15 by Daniel Stokes, last changed 2016-06-18 17:17 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
loopback_fast_path.patch Daniel Stokes, 2016-03-11 03:15 review
loopback_fast_path_v2.patch Daniel Stokes, 2016-05-18 04:52 review
loopback_fast_path_test.zip Daniel Stokes, 2016-05-19 04:52
loopback_fast_path_v3.patch Daniel Stokes, 2016-05-20 01:53 review
Messages (16)
msg261539 - (view) Author: Daniel Stokes (Daniel Stokes) * Date: 2016-03-11 03:15
Adding the SIO_LOOPBACK_FAST_PATH option to socket.ioctl on Windows allows Windows users to enable the loopback fast path option available on Windows 8+. This allows for much better TCP loopback performance on Windows.

For more information on TCP Loopback Fast Path, see:
  * https://msdn.microsoft.com/en-us/library/windows/desktop/jj841212%28v=vs.85%29.aspx
  * http://blogs.technet.com/b/wincat/archive/2012/12/05/fast-tcp-loopback-performance-and-low-latency-with-windows-server-2012-tcp-loopback-fast-path.aspx
msg264326 - (view) Author: Daniel Stokes (Daniel Stokes) * Date: 2016-04-26 20:21
The "Lifecycle of a Patch" document recommends pinging an issue after a month of no review. I do not see any special ping option, so I am assuming that means to post a comment.

To help with code review I tried to keep this change very similar to a previous similar change: http://bugs.python.org/issue6971
msg264335 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-04-27 00:53
Thanks for pinging, Daniel. I'm not a Windows user so I can't test the patch, but I left some review comments on Rietveld: http://bugs.python.org/review/26536/
msg265805 - (view) Author: Daniel Stokes (Daniel Stokes) * Date: 2016-05-18 04:52
Thank you for the review. I have made the changes suggested in the code and I have uploaded a new patch.
msg265845 - (view) Author: Daniel Stokes (Daniel Stokes) * Date: 2016-05-19 04:52
I have added a zip containing a simple server and client to test the patch with. With loopback fast path enabled, I get 3-5x more throughput on my Windows 10 computer with an AMD 900 series chipset.
msg265847 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-05-19 05:36
Thanks for the test script. I left some minor comments about loopback_fast_path_v2.patch on Rietveld.
msg265906 - (view) Author: Daniel Stokes (Daniel Stokes) * Date: 2016-05-20 01:53
I have uploaded a new patch with the suggested changes, thanks for the review.
msg268738 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-17 19:52
New changeset f8957c755c7a by Steve Dower in branch 'default':
Issue #26536: socket.ioctl now supports SIO_LOOPBACK_FAST_PATH. Patch by Daniel Stokes.
https://hg.python.org/cpython/rev/f8957c755c7a
msg268739 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-06-17 19:53
All looked good to me, and as far as I could see all of Berker's feedback was addressed, so consider it in!
msg268777 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-18 09:02
This is failing on Windows 7 buildbots:

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/7849/steps/test/logs/stdio
======================================================================
ERROR: test_sio_loopback_fast_path (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_socket.py", line 1226, in test_sio_loopback_fast_path
    s.ioctl(socket.SIO_LOOPBACK_FAST_PATH, True)
OSError: [WinError 10045] The attempted operation is not supported for the type of object referenced
msg268801 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-18 13:09
New changeset e8b10ed64e63 by Berker Peksag in branch 'default':
Issue #26536: Skip test_sio_loopback_fast_path under Windows 7
https://hg.python.org/cpython/rev/e8b10ed64e63
msg268802 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-06-18 13:40
It looks like SIO_LOOPBACK_FAST_PATH is also defined in older Windows versions. I updated the test to skip if SIO_LOOPBACK_FAST_PATH is defined and exc.winerror is WSAEOPNOTSUPP. Buildbot is now green: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/7855

Please reopen if I did something wrong.
msg268803 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-18 13:42
New changeset 213c240cce0a by Berker Peksag in branch 'default':
Issue #26536: Use spaces instead of tabs
https://hg.python.org/cpython/rev/213c240cce0a
msg268805 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-06-18 14:02
That'll handle the test fine, though I wonder whether we should try and conditionally define the constant at runtime?

Probably at least want to add a note to the docs about Win 8 being the minimum, especially since so many people are still on 7.
msg268806 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-06-18 14:18
> That'll handle the test fine, though I wonder whether we should try and conditionally define the constant at runtime?

+1. Is it safe to use IsWindows8OrGreater()?
msg268816 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-06-18 17:17
That's the best function to use.
History
Date User Action Args
2016-06-18 17:17:47steve.dowersetmessages: + msg268816
2016-06-18 14:18:49berker.peksagsetmessages: + msg268806
2016-06-18 14:02:25steve.dowersetmessages: + msg268805
2016-06-18 13:42:43python-devsetmessages: + msg268803
2016-06-18 13:40:54berker.peksagsetstatus: open -> closed

messages: + msg268802
2016-06-18 13:09:29python-devsetmessages: + msg268801
2016-06-18 09:02:34martin.pantersetstatus: closed -> open
nosy: + martin.panter
messages: + msg268777

2016-06-17 19:53:42steve.dowersetstatus: open -> closed
messages: + msg268739

assignee: steve.dower
resolution: fixed
stage: patch review -> resolved
2016-06-17 19:52:43python-devsetnosy: + python-dev
messages: + msg268738
2016-05-20 01:53:26Daniel Stokessetfiles: + loopback_fast_path_v3.patch

messages: + msg265906
2016-05-19 05:36:02berker.peksagsetmessages: + msg265847
2016-05-19 04:52:35Daniel Stokessetfiles: + loopback_fast_path_test.zip

messages: + msg265845
2016-05-18 04:52:56Daniel Stokessetfiles: + loopback_fast_path_v2.patch

messages: + msg265805
2016-04-27 00:53:50berker.peksagsetcomponents: + Windows
2016-04-27 00:53:22berker.peksagsetnosy: + berker.peksag
messages: + msg264335
components: + Extension Modules, - Windows, IO
2016-04-26 20:21:22Daniel Stokessetmessages: + msg264326
2016-03-11 16:46:58SilentGhostsetnosy: + pitrou, paul.moore, tim.golden, zach.ware, steve.dower

components: + Windows
stage: patch review
2016-03-11 03:15:28Daniel Stokescreate