Title: TimedRotatingFileHandler overwrite log
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: Harry-Lees, aeg, vinay.sajip
Priority: normal Keywords: patch

Created on 2021-05-20 10:21 by aeg, last changed 2021-06-06 07:21 by rhettinger.

File name Uploaded Description Edit aeg, 2021-05-20 10:50
Pull Requests
URL Status Linked Edit
PR 26553 open Harry-Lees, 2021-06-05 23:12
Messages (2)
msg394009 - (view) Author: Eiji Ito (aeg) Date: 2021-05-20 10:21
If you use TimedRotatingFileHandler and specify when='midnight',atTime, the log will always rollover if you start it at the same time as atTime.

For example, if atTime='00:00:00', and you run a script that outputs logs using the logger library twice at 00:00:00, the log that was rolled over the first time will be overwritten by the second rollover, and the 1st rollovered log will be lost.
msg395141 - (view) Author: Harry (Harry-Lees) * Date: 2021-06-04 23:39
This bug seems to come from the fact that the file rollover occurs it removes any files with same name as the destination file.

I believe this bug occurs on all "when" options, but the midnight option truncates the filename so only the date so it's much more likely to be a collision in filename so it's much easier to notice that it's happening.

One way of fixing this is to check if the destination file already exists, and if it does, append the logs to the end of the file instead of overwriting it.

Another potential fix is to check if the destination file already exists, and if it does rename it to something else, or even to choose a different name for the destination file of the log e.g. if the filename would be log.2021-05-05 but there is a collision, call the file log.2021-05-05(1).

Personally I prefer the last option, it seems like the best way to avoid any future issues that could arise from appending. I can open a PR for this in the morning.
Date User Action Args
2021-06-06 07:21:05rhettingersetassignee: vinay.sajip

nosy: + vinay.sajip
2021-06-05 23:12:47Harry-Leessetkeywords: + patch
stage: patch review
pull_requests: + pull_request25140
2021-06-04 23:39:51Harry-Leessetnosy: + Harry-Lees
messages: + msg395141
2021-05-20 10:50:45aegsetfiles: +
2021-05-20 10:21:59aegcreate