classification
Title: RotatingFileHandler with multi processes creates too small backup files
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: python-dev, vinay.sajip, yotam
Priority: normal Keywords: patch

Created on 2021-02-02 19:58 by yotam, last changed 2021-02-05 10:12 by vinay.sajip. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 24442 closed python-dev, 2021-02-04 17:51
Messages (2)
msg386165 - (view) Author: Yotam Medini (yotam) * Date: 2021-02-02 19:58
Using RotatingFileHandler by multi-processes
when they reach a point for rotation (roll-over)
they continue on separate files.

Two problems:
1. Eventually some backup files are left with sizes
   much less than the maxBytes configuration.
2. Intertwining events are not logged together,
   but separated by processes.
msg386521 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-02-05 10:12
This is not a bug. See

https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes

You should use SocketHandler or QueueHandler to deal with a situation where multiple processes log to files (whether rotating or not).
History
Date User Action Args
2021-02-05 10:12:23vinay.sajipsetstatus: open -> closed

nosy: + vinay.sajip
messages: + msg386521

resolution: not a bug
stage: patch review -> resolved
2021-02-04 17:51:00python-devsetkeywords: + patch
nosy: + python-dev

pull_requests: + pull_request23248
stage: patch review
2021-02-02 19:58:28yotamcreate