This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ambiguous _max_size parameter in SpooledTemporaryFile
Type: behavior Stage:
Components: Documentation, Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: brett.cannon, docs@python, jcc2220, lurchman
Priority: normal Keywords:

Created on 2018-08-20 19:47 by jcc2220, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg323809 - (view) Author: jcc2220 (jcc2220) Date: 2018-08-20 19:47
When _max_size is set to 0, the _check method will never call rollover, as the conditional for a rollover is never satisfied.  However, in the truncate method, the _max_size is not checked against 0, and so a rollover could be called when it is 0.  This is more of an issue of consistency - should 0 mean that it will never rollover?  If so, truncate should be modified.  Should 0 be interpreted as 'always rollover'?  If so, the _check should be modified.  Personally, I'd rather have something like -1 mean never, 0 mean always, and >0 mean only when greater than specified size.


John
msg323845 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-08-21 21:57
Do realize that anything which starts with an underscore, like _check, is considered a private API.
msg323850 - (view) Author: jcc2220 (jcc2220) Date: 2018-08-21 22:53
Here's an example of the inconsistency - if SpooledTemporaryFile is instantiated with max_size set to 0, it will not rollover, EXCEPT when truncate method is called.  Both write and writelines (public methods) call _check, which deals with rollover, but _check will never rollover if max_size is set to 0.
msg347475 - (view) Author: Andrew Scheller (lurchman) * Date: 2019-07-07 15:58
I agree that this is ambiguous behaviour. The docs at https://docs.python.org/3/library/tempfile.html#tempfile.SpooledTemporaryFile say "This function operates exactly as TemporaryFile() does, except that data is spooled in memory until the file size exceeds max_size, or until the file’s fileno() method is called, at which point the contents are written to disk and operation proceeds as with TemporaryFile().", and as the default value for max_size is 0, that would imply to me that _any_ data written to a SpooledTemporaryFile (constructed with a max_size of 0) would instantly get (internally) converted to a TemporaryFile.

Whereas looking at the code https://github.com/python/cpython/blob/master/Lib/tempfile.py#L650 it seems a max_size of 0 means "don't rollover". Perhaps it would have made sense to have SpooledTemporaryFile default to a max_size of None (and use that to mean "never rollover") ?

So as well as the inconsistency between rollover behaviour in the write()/writelines() methods and the truncate() method (when max_size is 0) that jcc2220 pointed out, I believe there's also a documentation issue here in that it's not clear what a max_size of 0 is /supposed/ to do.
History
Date User Action Args
2022-04-11 14:59:04adminsetgithub: 78627
2019-07-07 15:58:26lurchmansetversions: + Python 2.7, Python 3.5, Python 3.6, Python 3.8, Python 3.9
nosy: + lurchman, docs@python

messages: + msg347475

assignee: docs@python
components: + Documentation
2018-08-21 22:53:04jcc2220setmessages: + msg323850
2018-08-21 21:57:59brett.cannonsetnosy: + brett.cannon
messages: + msg323845
2018-08-20 19:47:35jcc2220create