Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(2)

Side by Side Diff: Doc/howto/logging-cookbook.rst

Issue 18759: Fix internal doc references for logging package (Closed)
Patch Set: Created 6 years, 7 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | Doc/howto/logging.rst » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 .. _logging-cookbook: 1 .. _logging-cookbook:
2 2
3 ================ 3 ================
4 Logging Cookbook 4 Logging Cookbook
5 ================ 5 ================
6 6
7 :Author: Vinay Sajip <vinay_sajip at red-dove dot com> 7 :Author: Vinay Sajip <vinay_sajip at red-dove dot com>
8 8
9 This page contains a number of recipes related to logging, which have been found 9 This page contains a number of recipes related to logging, which have been found
10 useful in the past. 10 useful in the past.
(...skipping 79 matching lines...) Expand 10 before | Expand all | Expand 10 after
90 2005-03-23 23:47:11,671 - spam_application - INFO - 90 2005-03-23 23:47:11,671 - spam_application - INFO -
91 calling auxiliary_module.some_function() 91 calling auxiliary_module.some_function()
92 2005-03-23 23:47:11,672 - spam_application.auxiliary - INFO - 92 2005-03-23 23:47:11,672 - spam_application.auxiliary - INFO -
93 received a call to 'some_function' 93 received a call to 'some_function'
94 2005-03-23 23:47:11,673 - spam_application - INFO - 94 2005-03-23 23:47:11,673 - spam_application - INFO -
95 done with auxiliary_module.some_function() 95 done with auxiliary_module.some_function()
96 96
97 Multiple handlers and formatters 97 Multiple handlers and formatters
98 -------------------------------- 98 --------------------------------
99 99
100 Loggers are plain Python objects. The :func:`addHandler` method has no minimum 100 Loggers are plain Python objects. The :meth:`~Logger.addHandler` method has no
101 minimum
101 or maximum quota for the number of handlers you may add. Sometimes it will be 102 or maximum quota for the number of handlers you may add. Sometimes it will be
102 beneficial for an application to log all messages of all severities to a text 103 beneficial for an application to log all messages of all severities to a text
103 file while simultaneously logging errors or above to the console. To set this 104 file while simultaneously logging errors or above to the console. To set this
104 up, simply configure the appropriate handlers. The logging calls in the 105 up, simply configure the appropriate handlers. The logging calls in the
105 application code will remain unchanged. Here is a slight modification to the 106 application code will remain unchanged. Here is a slight modification to the
106 previous simple module-based configuration example:: 107 previous simple module-based configuration example::
107 108
108 import logging 109 import logging
109 110
110 logger = logging.getLogger('simple_example') 111 logger = logging.getLogger('simple_example')
(...skipping 341 matching lines...) Expand 10 before | Expand all | Expand 10 after
452 453
453 About to start TCP server... 454 About to start TCP server...
454 59 root INFO Jackdaws love my big sphinx of quartz. 455 59 root INFO Jackdaws love my big sphinx of quartz.
455 59 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim. 456 59 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
456 69 myapp.area1 INFO How quickly daft jumping zebras vex. 457 69 myapp.area1 INFO How quickly daft jumping zebras vex.
457 69 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack. 458 69 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
458 69 myapp.area2 ERROR The five boxing wizards jump quickly. 459 69 myapp.area2 ERROR The five boxing wizards jump quickly.
459 460
460 Note that there are some security issues with pickle in some scenarios. If 461 Note that there are some security issues with pickle in some scenarios. If
461 these affect you, you can use an alternative serialization scheme by overriding 462 these affect you, you can use an alternative serialization scheme by overriding
462 the :meth:`makePickle` method and implementing your alternative there, as 463 the :meth:`~SocketHandler.makePickle` method and implementing your alternative
464 there, as
463 well as adapting the above script to use your alternative serialization. 465 well as adapting the above script to use your alternative serialization.
464 466
465 467
466 .. _context-info: 468 .. _context-info:
467 469
468 Adding contextual information to your logging output 470 Adding contextual information to your logging output
469 ---------------------------------------------------- 471 ----------------------------------------------------
472
473 .. currentmodule:: logging
470 474
471 Sometimes you want logging output to contain contextual information in 475 Sometimes you want logging output to contain contextual information in
472 addition to the parameters passed to the logging call. For example, in a 476 addition to the parameters passed to the logging call. For example, in a
473 networked application, it may be desirable to log client-specific information 477 networked application, it may be desirable to log client-specific information
474 in the log (e.g. remote client's username, or IP address). Although you could 478 in the log (e.g. remote client's username, or IP address). Although you could
475 use the *extra* parameter to achieve this, it's not always convenient to pass 479 use the *extra* parameter to achieve this, it's not always convenient to pass
476 the information in this way. While it might be tempting to create 480 the information in this way. While it might be tempting to create
477 :class:`Logger` instances on a per-connection basis, this is not a good idea 481 :class:`Logger` instances on a per-connection basis, this is not a good idea
478 because these instances are not garbage collected. While this is not a problem 482 because these instances are not garbage collected. While this is not a problem
479 in practice, when the number of :class:`Logger` instances is dependent on the 483 in practice, when the number of :class:`Logger` instances is dependent on the
(...skipping 22 matching lines...) Expand all
502 :class:`LoggerAdapter`:: 506 :class:`LoggerAdapter`::
503 507
504 def debug(self, msg, *args, **kwargs): 508 def debug(self, msg, *args, **kwargs):
505 """ 509 """
506 Delegate a debug call to the underlying logger, after adding 510 Delegate a debug call to the underlying logger, after adding
507 contextual information from this adapter instance. 511 contextual information from this adapter instance.
508 """ 512 """
509 msg, kwargs = self.process(msg, kwargs) 513 msg, kwargs = self.process(msg, kwargs)
510 self.logger.debug(msg, *args, **kwargs) 514 self.logger.debug(msg, *args, **kwargs)
511 515
512 The :meth:`process` method of :class:`LoggerAdapter` is where the contextual 516 The :meth:`~LoggerAdapter.process` method of :class:`LoggerAdapter` is where
517 the contextual
513 information is added to the logging output. It's passed the message and 518 information is added to the logging output. It's passed the message and
514 keyword arguments of the logging call, and it passes back (potentially) 519 keyword arguments of the logging call, and it passes back (potentially)
515 modified versions of these to use in the call to the underlying logger. The 520 modified versions of these to use in the call to the underlying logger. The
516 default implementation of this method leaves the message alone, but inserts 521 default implementation of this method leaves the message alone, but inserts
517 an 'extra' key in the keyword argument whose value is the dict-like object 522 an 'extra' key in the keyword argument whose value is the dict-like object
518 passed to the constructor. Of course, if you had passed an 'extra' keyword 523 passed to the constructor. Of course, if you had passed an 'extra' keyword
519 argument in the call to the adapter, it will be silently overwritten. 524 argument in the call to the adapter, it will be silently overwritten.
520 525
521 The advantage of using 'extra' is that the values in the dict-like object are 526 The advantage of using 'extra' is that the values in the dict-like object are
522 merged into the :class:`LogRecord` instance's __dict__, allowing you to use 527 merged into the :class:`LogRecord` instance's __dict__, allowing you to use
523 customized strings with your :class:`Formatter` instances which know about 528 customized strings with your :class:`Formatter` instances which know about
524 the keys of the dict-like object. If you need a different method, e.g. if you 529 the keys of the dict-like object. If you need a different method, e.g. if you
525 want to prepend or append the contextual information to the message string, 530 want to prepend or append the contextual information to the message string,
526 you just need to subclass :class:`LoggerAdapter` and override :meth:`process` 531 you just need to subclass :class:`LoggerAdapter` and override
527 to do what you need. Here is a simple example:: 532 :meth:`~LoggerAdapter.process` to do what you need. Here is a simple example::
528 533
529 class CustomAdapter(logging.LoggerAdapter): 534 class CustomAdapter(logging.LoggerAdapter):
530 """ 535 """
531 This example adapter expects the passed in dict-like object to have a 536 This example adapter expects the passed in dict-like object to have a
532 'connid' key, whose value in brackets is prepended to the log message. 537 'connid' key, whose value in brackets is prepended to the log message.
533 """ 538 """
534 def process(self, msg, kwargs): 539 def process(self, msg, kwargs):
535 return '[%s] %s' % (self.extra['connid'], msg), kwargs 540 return '[%s] %s' % (self.extra['connid'], msg), kwargs
536 541
537 which you can use like this:: 542 which you can use like this::
(...skipping 88 matching lines...) Expand 10 before | Expand all | Expand 10 after
626 .. _multiple-processes: 631 .. _multiple-processes:
627 632
628 Logging to a single file from multiple processes 633 Logging to a single file from multiple processes
629 ------------------------------------------------ 634 ------------------------------------------------
630 635
631 Although logging is thread-safe, and logging to a single file from multiple 636 Although logging is thread-safe, and logging to a single file from multiple
632 threads in a single process *is* supported, logging to a single file from 637 threads in a single process *is* supported, logging to a single file from
633 *multiple processes* is *not* supported, because there is no standard way to 638 *multiple processes* is *not* supported, because there is no standard way to
634 serialize access to a single file across multiple processes in Python. If you 639 serialize access to a single file across multiple processes in Python. If you
635 need to log to a single file from multiple processes, one way of doing this is 640 need to log to a single file from multiple processes, one way of doing this is
636 to have all the processes log to a :class:`SocketHandler`, and have a separate 641 to have all the processes log to a :class:`~handlers.SocketHandler`, and have a
642 separate
637 process which implements a socket server which reads from the socket and logs 643 process which implements a socket server which reads from the socket and logs
638 to file. (If you prefer, you can dedicate one thread in one of the existing 644 to file. (If you prefer, you can dedicate one thread in one of the existing
639 processes to perform this function.) :ref:`This section <network-logging>` 645 processes to perform this function.) :ref:`This section <network-logging>`
640 documents this approach in more detail and includes a working socket receiver 646 documents this approach in more detail and includes a working socket receiver
641 which can be used as a starting point for you to adapt in your own 647 which can be used as a starting point for you to adapt in your own
642 applications. 648 applications.
643 649
644 If you are using a recent version of Python which includes the 650 If you are using a recent version of Python which includes the
645 :mod:`multiprocessing` module, you could write your own handler which uses the 651 :mod:`multiprocessing` module, you could write your own handler which uses the
646 :class:`Lock` class from this module to serialize access to the file from 652 :class:`~multiprocessing.Lock` class from this module to serialize access to
653 the file from
647 your processes. The existing :class:`FileHandler` and subclasses do not make 654 your processes. The existing :class:`FileHandler` and subclasses do not make
648 use of :mod:`multiprocessing` at present, though they may do so in the future. 655 use of :mod:`multiprocessing` at present, though they may do so in the future.
649 Note that at present, the :mod:`multiprocessing` module does not provide 656 Note that at present, the :mod:`multiprocessing` module does not provide
650 working lock functionality on all platforms (see 657 working lock functionality on all platforms (see
651 http://bugs.python.org/issue3770). 658 http://bugs.python.org/issue3770).
652 659
653 .. currentmodule:: logging.handlers 660 Alternatively, you can use a ``Queue`` and a :class:`~handlers.QueueHandler`
654 661 to send
655 Alternatively, you can use a ``Queue`` and a :class:`QueueHandler` to send
656 all logging events to one of the processes in your multi-process application. 662 all logging events to one of the processes in your multi-process application.
657 The following example script demonstrates how you can do this; in the example 663 The following example script demonstrates how you can do this; in the example
658 a separate listener process listens for events sent by other processes and logs 664 a separate listener process listens for events sent by other processes and logs
659 them according to its own logging configuration. Although the example only 665 them according to its own logging configuration. Although the example only
660 demonstrates one way of doing it (for example, you may want to use a listener 666 demonstrates one way of doing it (for example, you may want to use a listener
661 thread rather than a separate listener process -- the implementation would be 667 thread rather than a separate listener process -- the implementation would be
662 analogous) it does allow for completely different logging configurations for 668 analogous) it does allow for completely different logging configurations for
663 the listener and the other processes in your application, and can be used as 669 the listener and the other processes in your application, and can be used as
664 the basis for code meeting your own specific requirements:: 670 the basis for code meeting your own specific requirements::
665 671
(...skipping 205 matching lines...) Expand 10 before | Expand all | Expand 10 after
871 Using file rotation 877 Using file rotation
872 ------------------- 878 -------------------
873 879
874 .. sectionauthor:: Doug Hellmann, Vinay Sajip (changes) 880 .. sectionauthor:: Doug Hellmann, Vinay Sajip (changes)
875 .. (see <http://blog.doughellmann.com/2007/05/pymotw-logging.html>) 881 .. (see <http://blog.doughellmann.com/2007/05/pymotw-logging.html>)
876 882
877 Sometimes you want to let a log file grow to a certain size, then open a new 883 Sometimes you want to let a log file grow to a certain size, then open a new
878 file and log to that. You may want to keep a certain number of these files, and 884 file and log to that. You may want to keep a certain number of these files, and
879 when that many files have been created, rotate the files so that the number of 885 when that many files have been created, rotate the files so that the number of
880 files and the size of the files both remain bounded. For this usage pattern, the 886 files and the size of the files both remain bounded. For this usage pattern, the
881 logging package provides a :class:`RotatingFileHandler`:: 887 logging package provides a :class:`~handlers.RotatingFileHandler`::
882 888
883 import glob 889 import glob
884 import logging 890 import logging
885 import logging.handlers 891 import logging.handlers
886 892
887 LOG_FILENAME = 'logging_rotatingfile_example.out' 893 LOG_FILENAME = 'logging_rotatingfile_example.out'
888 894
889 # Set up a specific logger with our desired output level 895 # Set up a specific logger with our desired output level
890 my_logger = logging.getLogger('MyLogger') 896 my_logger = logging.getLogger('MyLogger')
891 my_logger.setLevel(logging.DEBUG) 897 my_logger.setLevel(logging.DEBUG)
(...skipping 277 matching lines...) Expand 10 before | Expand all | Expand 10 after
1169 However, it should be borne in mind that each link in the chain adds run-time 1175 However, it should be borne in mind that each link in the chain adds run-time
1170 overhead to all logging operations, and the technique should only be used when 1176 overhead to all logging operations, and the technique should only be used when
1171 the use of a :class:`Filter` does not provide the desired result. 1177 the use of a :class:`Filter` does not provide the desired result.
1172 1178
1173 1179
1174 .. _zeromq-handlers: 1180 .. _zeromq-handlers:
1175 1181
1176 Subclassing QueueHandler - a ZeroMQ example 1182 Subclassing QueueHandler - a ZeroMQ example
1177 ------------------------------------------- 1183 -------------------------------------------
1178 1184
1179 You can use a :class:`QueueHandler` subclass to send messages to other kinds 1185 You can use a :class:`~handlers.QueueHandler` subclass to send messages to
1186 other kinds
1180 of queues, for example a ZeroMQ 'publish' socket. In the example below,the 1187 of queues, for example a ZeroMQ 'publish' socket. In the example below,the
1181 socket is created separately and passed to the handler (as its 'queue'):: 1188 socket is created separately and passed to the handler (as its 'queue')::
1182 1189
1183 import zmq # using pyzmq, the Python binding for ZeroMQ 1190 import zmq # using pyzmq, the Python binding for ZeroMQ
1184 import json # for serializing records portably 1191 import json # for serializing records portably
1185 1192
1186 ctx = zmq.Context() 1193 ctx = zmq.Context()
1187 sock = zmq.Socket(ctx, zmq.PUB) # or zmq.PUSH, or other suitable value 1194 sock = zmq.Socket(ctx, zmq.PUB) # or zmq.PUSH, or other suitable value
1188 sock.bind('tcp://*:5556') # or wherever 1195 sock.bind('tcp://*:5556') # or wherever
1189 1196
(...skipping 19 matching lines...) Expand all
1209 data = json.dumps(record.__dict__) 1216 data = json.dumps(record.__dict__)
1210 self.queue.send(data) 1217 self.queue.send(data)
1211 1218
1212 def close(self): 1219 def close(self):
1213 self.queue.close() 1220 self.queue.close()
1214 1221
1215 1222
1216 Subclassing QueueListener - a ZeroMQ example 1223 Subclassing QueueListener - a ZeroMQ example
1217 -------------------------------------------- 1224 --------------------------------------------
1218 1225
1219 You can also subclass :class:`QueueListener` to get messages from other kinds 1226 You can also subclass :class:`~handlers.QueueListener` to get messages from
1227 other kinds
1220 of queues, for example a ZeroMQ 'subscribe' socket. Here's an example:: 1228 of queues, for example a ZeroMQ 'subscribe' socket. Here's an example::
1221 1229
1222 class ZeroMQSocketListener(QueueListener): 1230 class ZeroMQSocketListener(QueueListener):
1223 def __init__(self, uri, *handlers, **kwargs): 1231 def __init__(self, uri, *handlers, **kwargs):
1224 self.ctx = kwargs.get('ctx') or zmq.Context() 1232 self.ctx = kwargs.get('ctx') or zmq.Context()
1225 socket = zmq.Socket(self.ctx, zmq.SUB) 1233 socket = zmq.Socket(self.ctx, zmq.SUB)
1226 socket.setsockopt(zmq.SUBSCRIBE, '') # subscribe to everything 1234 socket.setsockopt(zmq.SUBSCRIBE, '') # subscribe to everything
1227 socket.connect(uri) 1235 socket.connect(uri)
1228 1236
1229 def dequeue(self): 1237 def dequeue(self):
(...skipping 454 matching lines...) Expand 10 before | Expand all | Expand 10 after
1684 if __name__ == '__main__': 1692 if __name__ == '__main__':
1685 main() 1693 main()
1686 1694
1687 When the above script is run, it prints:: 1695 When the above script is run, it prints::
1688 1696
1689 message 1 >>> {"snowman": "\u2603", "set_value": [1, 2, 3]} 1697 message 1 >>> {"snowman": "\u2603", "set_value": [1, 2, 3]}
1690 1698
1691 Note that the order of items might be different according to the version of 1699 Note that the order of items might be different according to the version of
1692 Python used. 1700 Python used.
1693 1701
OLDNEW
« no previous file with comments | « no previous file | Doc/howto/logging.rst » ('j') | no next file with comments »

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+