This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: logging RotatingFileHandler cause too long to fininsh
Type: performance Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: davy.zhang, vinay.sajip
Priority: normal Keywords:

Created on 2010-05-19 10:56 by davy.zhang, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (2)
msg106043 - (view) Author: davy zhang (davy.zhang) Date: 2010-05-19 10:56
the function below is used to determine the proper file name of the next log file. But the question is if I set a large number of self.backupCount like 10000000, it will take too long to calculate the right file name. I think it could be a better way to do this instead of finding from the end.

    def doRollover(self):
        """
        Do a rollover, as described in __init__().
        """

        self.stream.close()
        if self.backupCount > 0:
            for i in range(self.backupCount - 1, 0, -1):
                sfn = "%s.%d" % (self.baseFilename, i)
                dfn = "%s.%d" % (self.baseFilename, i + 1)
                if os.path.exists(sfn):
                    #print "%s -> %s" % (sfn, dfn)
                    if os.path.exists(dfn):
                        os.remove(dfn)
                    os.rename(sfn, dfn)
            dfn = self.baseFilename + ".1"
            if os.path.exists(dfn):
                os.remove(dfn)
            os.rename(self.baseFilename, dfn)
            #print "%s -> %s" % (self.baseFilename, dfn)
        self.mode = 'w'
        self.stream = self._open()
msg106063 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2010-05-19 13:21
If you look closely, the system is not determining the name of the next log file. It is renaming log files - app.log.2 -> app.log.3, app.log.1 -> app.log.2, app.log -> app.log.1. The "next log file" is always app.log (or whatever the base file name is, I've just used app.log as an example).

It doesn't make sense to set a very large backupCount value, say of ten million (as in your example); why would you want to do this? Did you perhaps mean 10,000,000 for the maxBytes value? If you did mean backupCount, and if you do have a valid reason, then you can subclass this handler and write your own rollover implementation which does whatever you want.
History
Date User Action Args
2022-04-11 14:57:01adminsetgithub: 53010
2010-05-19 13:21:15vinay.sajipsetstatus: open -> closed
resolution: not a bug
messages: + msg106063
2010-05-19 11:22:22pitrousetassignee: vinay.sajip

nosy: + vinay.sajip
2010-05-19 10:58:25davy.zhangsettype: performance
components: + Library (Lib)
2010-05-19 10:56:42davy.zhangcreate