Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the call of tempfile.NamedTemporaryFile fails and leaves a file on the disk #70573

Closed
EugeneViktorov mannequin opened this issue Feb 18, 2016 · 18 comments
Closed

the call of tempfile.NamedTemporaryFile fails and leaves a file on the disk #70573

EugeneViktorov mannequin opened this issue Feb 18, 2016 · 18 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@EugeneViktorov
Copy link
Mannequin

EugeneViktorov mannequin commented Feb 18, 2016

BPO 26385
Nosy @terryjreedy, @vstinner, @vadmium, @eryksun
Files
  • temp_file.py
  • issue26385.diff
  • issue26385_2.diff
  • issue26385_3.diff
  • issue26385_4_py2.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-02-29.19:17:40.733>
    created_at = <Date 2016-02-18.16:17:52.388>
    labels = ['type-bug', 'library']
    title = 'the call of tempfile.NamedTemporaryFile fails and leaves a file on the disk'
    updated_at = <Date 2016-02-29.19:17:40.732>
    user = 'https://bugs.python.org/EugeneViktorov'

    bugs.python.org fields:

    activity = <Date 2016-02-29.19:17:40.732>
    actor = 'SilentGhost'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-02-29.19:17:40.733>
    closer = 'SilentGhost'
    components = ['Library (Lib)']
    creation = <Date 2016-02-18.16:17:52.388>
    creator = 'Eugene Viktorov'
    dependencies = []
    files = ['41957', '41959', '41968', '41979', '42015']
    hgrepos = []
    issue_num = 26385
    keywords = ['patch']
    message_count = 18.0
    messages = ['260466', '260474', '260493', '260505', '260506', '260510', '260511', '260512', '260549', '260551', '260555', '260557', '260565', '260566', '260578', '260598', '260758', '261004']
    nosy_count = 7.0
    nosy_names = ['terry.reedy', 'vstinner', 'SilentGhost', 'python-dev', 'martin.panter', 'eryksun', 'Eugene Viktorov']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue26385'
    versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

    @EugeneViktorov
    Copy link
    Mannequin Author

    EugeneViktorov mannequin commented Feb 18, 2016

    When calling the tempfile.NamedTemporaryFile with mode='wr' or with any other wrong value for "mode", it raises the ValueError and silently leaves an unknown, just created file on the file system.

    @EugeneViktorov EugeneViktorov mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Feb 18, 2016
    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Feb 18, 2016

    Here is a naïve fix including a test.

    @vadmium
    Copy link
    Member

    vadmium commented Feb 19, 2016

    This looks like an extension of bpo-21058. Does the unlink() work on Windows? It seems to me that the file is configured to remove itself on close(), therefore I expect unlink() will raise an exception of its own. Also made some suggestions in the code review.

    This problem also affects Python 2, if you fudge the right wrong parameters:

    >>> NamedTemporaryFile((), prefix="blaua.")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/proj/python/cpython/Lib/tempfile.py", line 477, in NamedTemporaryFile
        file = _os.fdopen(fd, mode, bufsize)
    TypeError: argument 2 must be string, not tuple
    [59140 refs]
    >>> glob("/tmp/blaua.*")
    ['/tmp/blaua.AFtEqx']

    @eryksun
    Copy link
    Contributor

    eryksun commented Feb 19, 2016

    Does the unlink() work on Windows?

    Yes. O_TEMPORARY opens the file with FILE_SHARE_DELETE, so unlink won't raise an error.

    Opening a file creates and returns a handle for a kernel File object that references the underlying file/link/stream control block in the file system. There may be multiple open File objects from separate NtCreateFile and NtOpenFile system calls, but they all reference a common file control block. Deleting a file requires a File handle, which is used to set the delete disposition in the control block. When all references (handle and pointer) to all File objects that reference the file are closed, the file is unlinked if the delete disposition is set.

    The way delete-on-close works is to set a flag in the File object that causes the delete disposition to be automatically set when the File object is closed. However, any File handle that references the file can be used to set or unset this disposition if it has DELETE access. So it's harmless to manually call DeleteFile on the file (i.e. NtOpenFile and NtSetInformationFile to set the delete disposition) beforehand.

    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Feb 19, 2016

    I wonder if Victor could clarify why bare except wasn't used in the python3 version.

    Anyway, here is the updated patch testing for TypeError as well.

    @vstinner
    Copy link
    Member

    I wonder if Victor could clarify why bare except wasn't used in the python3 version.

    What do you call a "bare except"?

    I wrote "except Exception: <cleanup code>; raise", it doesn't ignore the error. I want to always call the cleanup code on error.

    @vadmium
    Copy link
    Member

    vadmium commented Feb 19, 2016

    Victor: You changed “except Exception:” to bare “except:” in revision 182f08c0dd45 in Python 2, but the Python 3 code never got such a change.

    @vstinner
    Copy link
    Member

    Victor: You changed “except Exception:” to bare “except:” in revision 182f08c0dd45 in Python 2, but the Python 3 code never got such a change.

    Ah strange :-) Python 3 must also be fixed.

    @terryjreedy
    Copy link
    Member

    `except:` is equivalent to `except BaseException:`. When I introduced the former into idlelib, a couple of years ago, I was told to do the latter in a follow-up patch, so that there would be no doubt as to the intent. The same should be done here.

    @vstinner
    Copy link
    Member

    I prefer "except:" over "except BaseException:". What is the benefit
    of passing explicitly BaseException?

    @vadmium
    Copy link
    Member

    vadmium commented Feb 20, 2016

    Eryk Sun: The patch proposes to add an unlink() call after the file has been closed:

    except Exception:
        _os.close(fd)  # This automatically deletes the file right?
        _os.unlink(name)  # Won’t this raise FileNotFoundError?
        raise

    By your explanation, it sounds like it would be better to call unlink() before close().

    Terry & Victor: Writing the explicit “except BaseException:” makes it clear you weren’t being lazy in figuring out what exceptions you want to catch. But in this case the “raise” at the end of the the exception handler make it clear enough for me. I would be happy with either option.

    @terryjreedy
    Copy link
    Member

    It makes it clearer that you know that it will 'everything' and intend to catch do so and that you did not casually toss in 'except:', as used to be the habit of many. There are over 20 bare excepts in idlelib and I suspect many or most are unintentionally over-broad.

    @eryksun
    Copy link
    Contributor

    eryksun commented Feb 20, 2016

    By your explanation, it sounds like it would be better
    to call unlink() before close().

    Sorry, I was responding in general, because I thought you meant unlink would fail like it would for most open files on Windows, because the CRT normally doesn't open files with delete sharing. But I see what you meant now. Yes, the order needs to be reversed as unlink() and then close() for this to work. Doing the close first does raise a FileNotFoundError.

    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Feb 20, 2016

    Here is the updated patch including fixes for except and order of deletion.

    @vadmium
    Copy link
    Member

    vadmium commented Feb 20, 2016

    I think patch 3 is good for Python 3, thankyou. For Python 2, the test will have to be adjusted. From memory, mode='wr' is accepted without an exception, and mode=2 triggers an early error (so we never trigger the bug).

    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Feb 21, 2016

    Then, I think the TypeError check could be dropped and 'wr' replaced by an obviously wrong value, both seem fairly trivial. I don't have a working 2.7 checkout, so if anyone wants to extend the fix to that branch, they're more than welcome.

    @vadmium
    Copy link
    Member

    vadmium commented Feb 24, 2016

    Here is my proposed version for Python 2.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 29, 2016

    New changeset 5bfb4147405e by Martin Panter in branch '2.7':
    Issue bpo-26385: Cleanup NamedTemporaryFile if fdopen() fails, by SilentGhost
    https://hg.python.org/cpython/rev/5bfb4147405e

    New changeset a1c125f21db4 by Martin Panter in branch '3.5':
    Issue bpo-26385: Cleanup NamedTemporaryFile if open() fails, by SilentGhost
    https://hg.python.org/cpython/rev/a1c125f21db4

    New changeset 865cf8eba51a by Martin Panter in branch 'default':
    Issue bpo-26385: Merge NamedTemporaryFile fix from 3.5
    https://hg.python.org/cpython/rev/865cf8eba51a

    @SilentGhost SilentGhost mannequin closed this as completed Feb 29, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants