Title: SocketServer.DatagramRequestHandler Broken under Linux
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.6, Python 3.5, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, giampaolo.rodola, jimd, martin.panter, python-dev, terry.reedy
Priority: normal Keywords: patch

Created on 2009-04-23 22:20 by jimd, last changed 2016-02-24 20:32 by martin.panter. This issue is now closed.

File name Uploaded Description Edit
bind-unix.patch martin.panter, 2016-02-19 02:55 review
Messages (7)
msg86380 - (view) Author: Jim Dennis (jimd) Date: 2009-04-23 22:20
.../lib/python2.*/ in class DatagramRequestHandler
contains the following comment:
    # XXX Regrettably, I cannot get this working on Linux;
    # s.recvfrom() doesn't return a meaningful client address.

This is a poor way to document the limitation.  Someone writing code and
carefully adhering to the documentation will be presented with an
exception that stems from this and forced to read source code for the
standard libraries to understand why it's not behaving as documented.

In Issue1767511 it was recommended that the finish() method only call
sendto() if wfile.getvalue() returns a meaningful value.  (The bug here
is that this returns None under Linux and the default handler calls
sendto on it anyway).

Perhaps it would be better to raise an exception with a clear error
message ("Not supported on this platform") and to update the standard
documentation to clarify the need to over-ride the .finish() method if
one does NOT want a response automatically sent back.

(The fact that finish() does a sendto() seems like a poor default for a
UNIX domain socket --- for use with the UnixDatagramServer class --- in
any event.  That leads to a loop in .server_forever() where the instance
is reading its own response).

In fact the whole question of whether the DatagramRequestHandler should
be used with a UnixDatagramServer is reasonable.  If it's a nonsensical
combination then perhaps some sort of check could be implemented to
raise an exception ... or at least a sensible alternative should be
added to the standard documentation.


#!/usr/bin/env python
import SocketServer
usock = '/var/tmp/pong'

class PongHandler(SocketServer.DatagramRequestHandler):
    '''Play ping pong over datagram sockets
    def handle(self):
        '''Respond to any request with "Pong"
    def finish(self):
        '''Necessary under Linux to prevent:
           File "/usr/lib/python2.5/", line 588, in finish
           self.socket.sendto(self.wfile.getvalue(), self.client_address)
           TypeError: argument must be string or read-only character
buffer, not
        if self.wfile.getvalue():

if __name__ == '__main__':
    SocketServer.UnixDatagramServer(usock, PongHandler).serve_forever()

... and testing this with:

#!/usr/bin/env python
import socket
USOCK = "/var/tmp/pong"
if __name__ == '__main__':
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
    sock.sendto("Ping", USOCK)
msg86381 - (view) Author: Jim Dennis (jimd) Date: 2009-04-23 22:53

What I said about the default sendto() in finish() causing a loop in
server_forever() was wrong.  I'd seen that behavior in an experimental
variation of the code.

The exceptions raised (and deficiencies in documentation) are the issue.
msg112665 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-03 20:20
Same comment is in 3.1.2

Jim, can you at least suggest a specific text change at a specific place for the doc? That would likely be applied.
msg259969 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-10 02:53
It is not that wfile.getvalue() returns None, it is that self.client_address is None. If the client explicitly binds to a name, then the server gets that name in the client_address, and then everything seems to work properly for me on Linux.

On Linux, using strace and an unbound anonymous client, I see that recvfrom() outputs a zero-length sending address structure, which Python’s makesockaddr() function converts to None with the comment /* No address -- may be recvfrom() from known socket */. The man page <> actually says the address length should be sizeof(sa_family_t) in this case, so maybe that is a minor bug for Linux or the man page.

How does this work on non-Linux platforms? I’m not really familiar with Unix domain sockets, especially datagram ones, but it seems to me that the main confusion is whether DatagramRequestHandler supports anonymous Unix domain clients or not.

As in Issue 1767511, I think DatagramRequestHandler is designed to always send a response back to the sender of the request. If you don’t want to send a response, why are you using this class?
msg260264 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-14 05:16
Also see the three commented-out tests in test_socketserver. I presume the code needs to ensure the client binds to a name, and then the tests will be able to receive something back from the server.
msg260499 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-02-19 02:50
Here is a patch to enable the Unix domain DatagramRequestHandler tests, and bind the client to a socket name. Tests pass for me on Linux :)

The Python 2 version will need to have lowercase socketserver changed case to camel-case SocketServer.
msg260760 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-02-24 06:30
New changeset 92ae4a305858 by Martin Panter in branch '2.7':
Issue #5824: Fix DatagramRequestHandler tests by binding the client socket

New changeset 0d9d8fdd9736 by Martin Panter in branch '3.5':
Issue #5824: Fix DatagramRequestHandler tests by binding the client socket

New changeset 113e9c6fd64d by Martin Panter in branch 'default':
Issue #5824: Merge socketserver tests from 3.5
Date User Action Args
2016-02-24 20:32:42martin.pantersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-02-24 06:30:20python-devsetnosy: + python-dev
messages: + msg260760
2016-02-19 02:55:03martin.pantersetfiles: + bind-unix.patch
2016-02-19 02:54:44martin.pantersetfiles: - bind-unix.patch
2016-02-19 02:50:24martin.pantersetfiles: + bind-unix.patch
versions: + Python 3.5, Python 3.6, - Python 3.1, Python 3.2
messages: + msg260499

components: + Tests, - Documentation
keywords: + patch
stage: needs patch -> patch review
2016-02-14 05:16:03martin.pantersetmessages: + msg260264
stage: needs patch
2016-02-10 02:53:51martin.pantersetnosy: + martin.panter
messages: + msg259969
2012-03-27 14:32:34giampaolo.rodolasetnosy: + giampaolo.rodola
2010-08-03 20:20:38terry.reedysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 2.5, Python 2.4
nosy: + terry.reedy, docs@python

messages: + msg112665

assignee: docs@python
components: + Documentation, - Library (Lib)
2009-04-23 22:53:35jimdsetmessages: + msg86381
2009-04-23 22:20:37jimdcreate