This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: New addition of vSockets to the python socket module
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Cathy Avery, berker.peksag, christian.heimes, gregory.p.smith, kushal.das, ncoghlan, r.david.murray
Priority: normal Keywords: patch

Created on 2016-07-21 15:36 by Cathy Avery, last changed 2022-04-11 14:58 by admin. This issue is now closed.

File name Uploaded Description Edit
vsocket.patch Cathy Avery, 2016-07-21 15:36 vsockets for linux support review
vsock_rev2.patch Cathy Avery, 2016-11-09 18:07 vsockets for linux support revision 2 review
nc-vsock Cathy Avery, 2016-12-19 13:09
0001-bpo-27584-New-addition-of-vSockets-to-the-python-soc.patch Cathy Avery, 2017-06-27 18:36
REAME.txt Cathy Avery, 2017-06-27 18:38
Pull Requests
URL Status Linked Edit
PR 2489 merged python-dev, 2017-06-29 14:59
Repositories containing patches
Messages (32)
msg270935 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-07-21 15:36
I have added AF_VSOCK support to python's 3.6 socket module ( socketmodule.c socketmodule.h cloned from ). The implementation is very similar to AF_NETLINK.  AF_VSOCK requires the VMware-specific VMCI transport which is currently upstream or the virtio-vsock drivers developed by Stefan Hajnoczi at Red Hat. The virtio-vsock drivers are not upstream yet but more information with source and build instructions can be found at

More information on vSocket programming can be found at

The VMCI transport supports SOCK_DGRAM and SOCK_STREAM on both Linux and Windows. Virtio-vsock currently supports SOCK_STREAM only on Linux.

My python addition supports SOCK_STREAM and SOCK_DGRAM calls on Linux only.

I have tested my implementation on both driver sets on Linux.

Attached is a diff file so you can see which files I've modified. These include a new I have already tested the new file generation by running autoreconf. 

Also included in the patch is an updated socket.rst file however I could not get the final html page to be double spaced.
msg272358 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-08-10 18:45
Looks like there's a missing versionadded directive in the doc patch.

Is it possible/sensible to add tests for the new feature?

(I haven't reviewed the patch in detail, hopefully someone with more experience with C socket programming than I have will do that.)
msg272363 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-08-10 19:16
Sure I can add tests. I would like to base them on the existing socket tests. Where are those?

I did add a version

+  .. versionadded:: 3.4

It just not may not be the right one.
msg272366 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-08-10 19:32
Ah, I see.  No, the versionadded will be 3.6, and should go *after* the documentation of the new socket type.

The existing socket tests are in Lib/test/
msg272403 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2016-08-11 06:46
The patch can be applied, and build successfully. I have ran the current test suite[1]. The two failed tests do not seem to be have anything to do with this patch (read the end of the consoleText output). I think the thing remaining is the new test cases, and NEWS file update. 

msg280423 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-11-09 18:07
Please forgive the long delay in providing this update. I got a little sidetracked. Attached is the patch for Python 3.7. It includes fixes suggested in rev 1 plus VSOCK tests in


msg282798 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-12-09 19:14
Is there anything else that is needed for this patch?

msg283534 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-12-18 03:31
The second patch seems to be missing the configure changes.  Also, the tests have some over-long lines (we limit line lengths to 79 characters).  I realize there are other long lines in that file, but no need to add more :)

There is trailing whitespace on a number of lines in your patch.

Since this is new, we may not want to accept it until the support hits upstream.  Specifically, it will be difficult to get a review if the reviewer has to build a custom kernel to test the code :)  You do say that the VMCI is upstream, but I don't know what that means.  Which upstream?

Note: I'm not familiar with the socket C code, so I haven't reviewed the C code changes.  The tests look fine to me.

For the docs, the proposal doesn't seem to follow the format of the existing docs.  I would expect only the first paragraph located where you have it.  The remaining constants should be in the 'module contents'/'constants' section, I think.  Yes, that means each one gets a '.. versionadded' label.  Presumably also an 'availablility' label with whatever the minimum kernel version is...another reason we may need to wait.
msg283535 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-12-18 03:37
Oh, I see, the ac changes are there, I was looking at the patch delta instead of the complete patch.
msg283607 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-12-19 11:55
Is there a format checker I could use on the patch?

VMCI and the vmw_vsock_vmci_transport kernel modules are located in the upstream linux tree at 


They have been there for about years. These drivers are part of various downstream kernels such as RHEL. You will need a Vmware virtual machine in order to test it. 

Only the virtio-vsock driver is a new vsock application that needs to be custom built.
msg283608 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-12-19 11:56
Sorry about the typo the drivers have been there for about 4 years.
msg283612 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-12-19 12:12
'make patchcheck' will do whitespace checking because that's hard to eyeball (although many editors/IDEs do support making it visible nowadays).  We don't use any other checking tools other than eyeballs, since not all of the existing code conforms to PEP7/8 and for various reasons we aren't going to update most of the old code to conform.

So, if I'm running an ubuntu virtual machine under VMWare Fusion (which I already have set up) I should be able to get the tests to run?  Or does it need to be RedHat (or presumably CentOS)?
msg283620 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2016-12-19 13:09
First make sure the driver is in your kernel. It will be with RHEL. Look in /lib/modeles/"your kernel name"/kernel/net/vmw_vsock/vmw_vsock_vmci_transport. I have never tried it on vmware fusion. I have tested it on ESX. See if there is a VMCI option to enable on your VM's settings. Start the vm and do an lsmod to see if vmw_vsock_vmci_transport is loaded.

I've attached a little C program thats netcat for vsock. Its a quick confirmation that your transport is loaded correctly. It will show you your CID.

run ./nc-vsock

CID = 973033371
CID = 0x39ff4f9b
usage: ./nc-vsock [-l <port> [-t <dst> <dstport>] | <cid> <port>]
msg296410 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-06-20 10:51
The vsock code is now in the linux upstream kernel and qemu. I will be resubmitting my vsock patches for python. So from what I can tell the tip of the devel tree is for 3.7 and that the source control has switched to git. My question is do I need a github account or can I push git patches directly to this issues page? 


msg296413 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-06-20 11:15
You don't need a GitHub account for contributing to CPython, but the pull request workflow is the preferred way. You can still attach your patches to this issue.
msg297064 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-06-27 18:36
I've attached the third version of VSOCK patch addressing the concerns of the last rev. I've also included a README file that lists instructions on how to setup a test environment.

msg297065 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-06-27 18:38
Help file.
msg297276 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-06-29 15:18
I also issued a pull request

Let me know if I screwed it up.


msg297285 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-06-29 17:10
The build and test bots failed but I don't understand why. The build could not find module _socket but that is not new and other modules failed.

The test could not import fcntl which I did add but should not fail as I have run this test many time and other tests import fcntl.
msg297516 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-07-02 13:04
I'm attempting to figure out whether or not we have a buildbot in the Buildbot fleet that will cover this test case.

Based on the pre-merge CI run, it seems Ubuntu 14.04 is too old to include the required kernel headers.

However, it looks like RHEL/CentOS are also currently still missing the userspace changes to fully enable AF_VSOCK support (as the Red Hat backport flow appears to have gone through the dedicated hypervisor variant first):

So it's looking to me like we're going to need either a recent Fedora, a non-LTS Ubuntu, or a Debian 9 system to be confident we have the right headers available.
msg297568 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-03 11:18
Fedora 25 has the proper headers.
msg297835 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-06 18:51
So I revised my code based on the reviews and I passed all the checks ... now what?


msg298165 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-11 17:29
I think we are waiting on confirmation that we have a buildbot that has the necessary headers.
msg298166 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-11 17:31
OK thanks!
msg298167 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-07-11 17:35
I updated my PGO buildbot to Debian 9 which should have them.

~$ grep AF_VSOCK /usr/include/*/*/*
/usr/include/x86_64-linux-gnu/bits/socket.h:#define AF_VSOCK    PF_VSOCK
msg298169 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-11 17:50
You will also need linux/vm_sockets.h in order to build.
msg298171 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-07-11 17:59
yep, linux/vm_sockets.h exists.  I believe it's kernel headers are from 4.9.
msg298173 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-11 18:12
That should do it.
msg298174 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-07-11 18:14
Everything compiles successfully on this host - configure detects the header has HAVE_LINUX_VM_SOCKETS_H set to 1.

test_socket passes on this host.  However since i'm not running with a /dev/vsock, the unittests (correctly) skip the new tests.

testCreateSocket (test.test_socket.BasicVSOCKTest) ... skipped 'VSOCK sockets required for this test.'
testCrucialConstants (test.test_socket.BasicVSOCKTest) ... skipped 'VSOCK sockets required for this test.'
testSocketBufferSize (test.test_socket.BasicVSOCKTest) ... skipped 'VSOCK sockets required for this test.'
testVSOCKConstants (test.test_socket.BasicVSOCKTest) ... skipped 'VSOCK sockets required for this test.'
testStream (test.test_socket.ThreadedVSOCKSocketStreamTest) ... skipped 'VSOCK sockets required for this test.'
msg298805 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-21 14:26

I believe I am waiting for a final review. Is there anything else I need to be doing at this point.


msg298953 - (view) Author: Cathy Avery (Cathy Avery) * Date: 2017-07-24 10:49
There is an outstanding review on my pull request at as there is an red X at changes requested by kushaldas and I believe I have made the necessary changes. 

Again please let me know if there is anything that I need to do as I am new to this process.


msg301527 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-06 22:18
New changeset effc12f8e9e20d0951d2ba5883587666bd8218e3 by Christian Heimes (caavery) in branch 'master':
bpo-27584: New addition of vSockets to the python socket module (#2489)
Date User Action Args
2022-04-11 14:58:34adminsetgithub: 71771
2017-09-06 22:18:41christian.heimessetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-09-06 22:18:12christian.heimessetnosy: + christian.heimes
messages: + msg301527
2017-07-24 10:49:27Cathy Averysetmessages: + msg298953
2017-07-21 14:26:04Cathy Averysetmessages: + msg298805
2017-07-11 18:14:09gregory.p.smithsetmessages: + msg298174
2017-07-11 18:12:07Cathy Averysetmessages: + msg298173
2017-07-11 17:59:05gregory.p.smithsetmessages: + msg298171
2017-07-11 17:50:21Cathy Averysetmessages: + msg298169
2017-07-11 17:35:58gregory.p.smithsetmessages: + msg298167
2017-07-11 17:31:54Cathy Averysetmessages: + msg298166
2017-07-11 17:29:34r.david.murraysetmessages: + msg298165
2017-07-06 18:51:20Cathy Averysetmessages: + msg297835
2017-07-03 11:18:10Cathy Averysetmessages: + msg297568
2017-07-02 13:04:08ncoghlansetnosy: + ncoghlan
messages: + msg297516
2017-06-29 17:10:44Cathy Averysetmessages: + msg297285
2017-06-29 15:18:02Cathy Averysetmessages: + msg297276
2017-06-29 14:59:43python-devsetpull_requests: + pull_request2548
2017-06-27 18:38:00Cathy Averysetfiles: + REAME.txt

messages: + msg297065
2017-06-27 18:36:58Cathy Averysetfiles: + 0001-bpo-27584-New-addition-of-vSockets-to-the-python-soc.patch

messages: + msg297064
2017-06-20 11:15:25berker.peksagsetnosy: + berker.peksag
messages: + msg296413
2017-06-20 10:51:36Cathy Averysetmessages: + msg296410
2016-12-19 13:09:07Cathy Averysetfiles: + nc-vsock

messages: + msg283620
2016-12-19 12:12:01r.david.murraysetmessages: + msg283612
2016-12-19 11:56:50Cathy Averysetmessages: + msg283608
2016-12-19 11:55:36Cathy Averysetmessages: + msg283607
2016-12-18 03:37:44r.david.murraysetmessages: + msg283535
2016-12-18 03:31:32r.david.murraysetmessages: + msg283534
2016-12-09 19:14:08Cathy Averysetmessages: + msg282798
2016-11-09 18:07:21Cathy Averysetfiles: + vsock_rev2.patch

messages: + msg280423
versions: + Python 3.7, - Python 3.6
2016-08-11 06:46:06kushal.dassetnosy: + kushal.das
messages: + msg272403
2016-08-10 19:32:34r.david.murraysetmessages: + msg272366
2016-08-10 19:16:29Cathy Averysetmessages: + msg272363
2016-08-10 18:45:17r.david.murraysetnosy: + r.david.murray

messages: + msg272358
stage: patch review
2016-07-21 15:36:30Cathy Averycreate