This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Clear buffer in MemoryHandler flush
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: penlect, vinay.sajip
Priority: normal Keywords: patch

Created on 2019-11-12 21:43 by penlect, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 17132 merged penlect, 2019-11-12 21:51
Messages (2)
msg356489 - (view) Author: Daniel Andersson (penlect) * Date: 2019-11-12 21:43
The `logging.handlers.MemoryHandler` has a method `flush` which clears the buffer by assigning an empty list literal:

self.buffer = []

This forces the buffer to be a list instance.
My suggestion is to clear the buffer like this instead:

self.buffer.clear()

In this way it would be possible to implement a custom buffer or use the `collections.deque` when subclassing `MemoryHandler`. At the moment you must copy-past the `flush` method and modify it accordingly in the subclass:

```
def flush(self):
    # (Docstring skipped)
    self.acquire()
    try:
        if self.target:
	    for record in self.buffer:
	        self.target.handle(record)
	    self.buffer = []  # <-- change to `self.buffer.clear()`
    finally:
        self.release()
```

Example where this change is useful
===================================
This example creates a DequeMemoryHandler which uses the `collections.deque` as a buffer. Only the latest `capacity` number of records will stored in the buffer. The buffer is then flushed if and only if a record has a level greater or equal to `logging.ERROR`.

```
import collections
import logging
from logging.handlers import MemoryHandler


class DequeMemoryHandler(MemoryHandler):

    def __init__(self, capacity, *args, **kwargs):
        super().__init__(capacity, *args, **kwargs)
        self.buffer = collections.deque(maxlen=capacity)

    def shouldFlush(self, record):
        return record.levelno >= self.flushLevel


handler = DequeMemoryHandler(capacity=5, target=logging.StreamHandler())
logging.basicConfig(level=logging.INFO, handlers=[handler])

for i in range(1, 21):
    logging.info('Spam %d', i)
    if i % 10 == 0:
        logging.error(f'---> Eggs {i}')
```


Desired output (with the change):
---------------------------------
Spam 7
Spam 8
Spam 9
Spam 10
---> Eggs 10
Spam 17
Spam 18
Spam 19
Spam 20
---> Eggs 20


Actual output (without the change):
-----------------------------------
Spam 7
Spam 8
Spam 9
Spam 10
---> Eggs 10
Spam 11
Spam 12
Spam 13
Spam 14
Spam 15
Spam 16
Spam 17
Spam 18
Spam 19
Spam 20
---> Eggs 20
msg356563 - (view) Author: Daniel Andersson (penlect) * Date: 2019-11-13 22:20
The suggested change has been merged. I'm closing this issue. Thank you Vinay Sajip for reviewing.
History
Date User Action Args
2022-04-11 14:59:23adminsetgithub: 82962
2019-11-13 22:20:29penlectsetstatus: open -> closed
resolution: fixed
messages: + msg356563

stage: patch review -> resolved
2019-11-12 21:51:34penlectsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16642
2019-11-12 21:43:12penlectcreate