Title: RotatingFileHandler.doRollover behave wrong vs. log4j's
Type: Stage:
Components: Library (Lib) Versions: Python 2.5
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: jinq0123, nnorwitz, vinay.sajip
Priority: normal Keywords:

Created on 2007-07-12 10:34 by jinq0123, last changed 2007-09-27 05:53 by vinay.sajip. This issue is now closed.

Messages (5)
msg32480 - (view) Author: Jin Qing (jinq0123) Date: 2007-07-12 10:34
RotatingFileHandler.doRollover() will raise exception on rename() and cause all the subsequent log messages lost.

But log4j is better. It ignore the rename() error and reset the log file.

I read "[ 979252 ] Trap OSError when calling RotatingFileHandler.doRollover ( )", and think David London (groodude)'s solution is closer to log4j.

At least, python logging implementation should follow log4j's behavior, or improve it, such as only discard the current one message. 

logging should be easy to use.
msg32481 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-07-13 03:47
Vinay, could you take a look?  Thanks.
msg32482 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2007-07-13 17:51
The system was modified in the fix for #979252 to call handleError when an error occurred during the emit() processing.

I am travelling and do not have access to my systems at the moment, so cannot commit to look at this just yet.

For now, users are free to subclass the RotatingFileHandler and override handleError however they want, including implementing groodude's solution, which (as he says in his comments) is a band-aid to cover unknown error conditions. Since there is no good reason for the rename to fail, I would suggest to the poster to see if he can find out why he is getting exception on rename; and I will give this some thought when I next have time, but that will not be just yet, and until then I suggest using the subclass/override approach.

I'll leave it open to remind me to look at it, but I believe the workarounds I have suggested are not unreasonable for now.
msg32483 - (view) Author: Jin Qing (jinq0123) Date: 2007-07-16 03:05
Calling handleError is a workaround, not the correct way of log4j.  This is Python, and should be easy to use. Subclass & override is a little hard to use. If it is required for every users to subclass or patch RotatingFileHandler to use it, so why not patch it in the released version? Everybody expect logging module has a behavior in common with log4j.

The exception of rename() is real. rename() fails if the source file is locked. I found UltraEdit will lock file. Others say tail -f will lock file too. 

In C++ or java, rename() just returns error, but Python raises exception and causes different logging behavior. Why not just ignore the exception of rename()? Or try rename() and finally open the log file?

By the way, log4j is much more robust on opening the log file. It will create the parent directories and retry if open failed. Python logging should learn from log4j.
msg56155 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2007-09-27 05:53
I have now had some more time to think about this issue. I don't believe
any changes are warranted, because "Errors should never pass silently.
Unless explicitly silenced." and since rename errors are usually due to
application- or environment-specific conditions, you need to handle
these in application code.

If logging continues to use the existing log file because renaming
fails, then it does not behave according to expectations - e.g. maximum
sizes on log files will not be honoured.

Likewise, logging does not attempt to use makedirs() to ensure that the
parent directory path is created first - a typo in the path would lead
to an unexpected location for the log file.

While Python logging borrowed a lot from log4j, it is far from a
straight port; the Zen of Python is different from the Zen of Java.
Date User Action Args
2007-09-27 05:53:27vinay.sajipsetstatus: open -> closed
resolution: not a bug
messages: + msg56155
2007-07-12 10:34:38jinq0123create