This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: logging.handlers.SysLogHandler with TCP support
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: enigma, r.david.murray, vinay.sajip
Priority: normal Keywords: patch

Created on 2009-10-08 22:58 by enigma, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
handlers.py.diff enigma, 2009-10-09 01:12 Patch with Requested Corrections
sockhand.diff vinay.sajip, 2009-10-09 21:44 Alternative patch
Messages (8)
msg93766 - (view) Author: Jr Aquino (enigma) Date: 2009-10-08 22:58
This patch is to address the deficiency in sending syslog messages to an 
external syslog server via tcp or udp.

Currently the handler only really supports udp unless it is being 
redirected locally through a unix socket.

The submitted patch allows for the standard udp SOCK_DGRAM behavior to 
continue being the default, but allows for a protocol object to be defined 
as tcp to allow for the alternate protocol.
msg93767 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-10-09 00:11
I'll leave it to Vinay to pronounce on the request and the validity of
the patch, but a couple of comments on the mechanics of the patch:  You
have some rogue whitespace on one of the blank lines.  More importantly,
you are adding a keyword parameter at the beginning of the list of
keywords, which would change the semantics if someone passed in
arguments positionally.  Finally, you should check out PEP 8 about
Python style...the line length should be less the 80, which means your
docstring addition needs to be reformatted, as well as the __init__ line
itself.

Thanks for contributing!

Oh, and while I set the stage to test needed it doesn't look like there
are any existing tests for SysLogHandler....
msg93768 - (view) Author: Jr Aquino (enigma) Date: 2009-10-09 01:12
Here is the patch with the requested syntactical corrections.
msg93784 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2009-10-09 12:03
Which syslog daemon are you using? There are some issues pending with syslog-ng and Python logging (see issue6444) and in general syslog over TCP is not necessarily all that reliable, see

http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html

The requested change is not difficult to make, but it will be difficult for me to test because in general end-to-end testing for network APIs is not there in the standard regression test suite, and I don't have a setup where I can test it independently. So I'm sorry to say I will not be able to give this a very high priority at the moment (unless another committer can test this).

Just curious (as no one has asked for this before) why you're using TCP, given that it doesn't eliminate message loss and it is slower and has less support than UDP?
msg93801 - (view) Author: Jr Aquino (enigma) Date: 2009-10-09 15:17
Thank you for responding so quickly Vinay.

I am using a multitude of syslog daemons, from syslog, syslog-ng, 
rsyslog, and several different proprietary SIEM/SEM Log archiving 
appliances.  I work in the security sector.

(Yes I have read Rainer before, its actually a big reason I am writing 
the tool that I am working on. Gaps in logs are a real world problem!) 

Regarding issue 6444, these users are attempting to use SysLogHandler to 
write to the local system's syslog sockets. A way of piggy backing on 
the configuration of the local systems logging daemon.

What I am actually doing is ignoring the local systems syslog, and 
sending the syslog packets directly to a remote syslog server. This is 
to replay syslog data that may have been previously lost due to 
connectivity outages.  My tool can also be used to send the data to an 
external server for forensic reasons.

I believe the answer to your last question also sheds light on your 
first question!

The reason that I am looking to add TCP is because a lot of new data 
center architectures are heavily utilizing tcp syslog in a chained / 
centralized environment.  I am also seeing a lot of preferential 
treatment of tcp syslog on logging appliances such as Loglogic.

I am sorry to hear that your test environment is lacking the regression 
suites that you need.

I do hope that another commiter can test for us.  Syslog is an old 
technology and I hope that more efforts like mine and Rainer's can help 
to identify and correct deficiencies in the design.
msg93810 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2009-10-09 21:44
@Jr Aquino: can you please test the attached alternative patch with all
the various syslog daemons in Unix domain, UDP and TCP socket
combinations, and post your results here? Thanks.

P.S. Also available colourised at http://gist.github.com/206380
msg93811 - (view) Author: Jr Aquino (enigma) Date: 2009-10-09 23:06
Vinay, tested on all syslog daemons/servers.  Works perfectly.

Thank you very much. I appreciate your time greatly.
msg93842 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2009-10-10 20:34
Fix checked into trunk and py3k.
History
Date User Action Args
2022-04-11 14:56:53adminsetgithub: 51335
2009-10-10 20:34:09vinay.sajipsetstatus: open -> closed
resolution: fixed
messages: + msg93842

stage: test needed ->
2009-10-09 23:06:56enigmasetstatus: pending -> open

messages: + msg93811
2009-10-09 21:44:56vinay.sajipsetstatus: open -> pending
assignee: vinay.sajip
messages: + msg93810

files: + sockhand.diff
2009-10-09 15:17:43enigmasetmessages: + msg93801
2009-10-09 12:03:05vinay.sajipsetmessages: + msg93784
2009-10-09 01:13:52enigmasetfiles: - handlers.py.diff
2009-10-09 01:12:01enigmasetfiles: + handlers.py.diff

messages: + msg93768
2009-10-09 00:11:54r.david.murraysetpriority: normal

type: behavior -> enhancement
components: + Library (Lib), - Extension Modules
versions: + Python 2.7, Python 3.2, - Python 2.6
nosy: + vinay.sajip, r.david.murray

messages: + msg93767
stage: test needed
2009-10-08 22:58:20enigmacreate