The security issues of `tempfile.mktemp` are clear when the return value is used to create a filesystem entry. The documentation and docstrings (and even some comments on past issues) are correct o deprecate its use for that purpose.

The function has a use which doers not have security implications: generating test data. When a test case wants to generate unpredictable, unique, valid filesystem paths – and *never access those paths* on the filesystem – the `tempfile.mktemp` function is right there and is very useful.

The `tempfile._RandomNameSequence` class would also be useful, but its name also makes clear that it is not part of the library public API.

Please make that functionality available for the purpose of *only* generating filesystem paths as `tempfile._RandomNameSequence` does, in a public, supported, non-deprecated API.

It has been pointed out that `tempfile.mktemp` does in fact access the filesystem, to query whether the entry exists.

So this request would be best met by exposing a simple “get a new return value from the `tempfile._RandomNameSequence` instance” function.

An example::

    import io
    import tempfile
    names = tempfile._get_candidate_names()

    def test_frobnicates_configured_spungfile():
        """ ‘foo’ should frobnicate the configured spungfile. """

        fake_file_path = os.path.join(tempfile.gettempdir(), names.next())
        fake_file = io.BytesIO("Lorem ipsum, dolor sit amet".encode("utf-8"))

        patch_builtins_open(
                when_accessing_path=fake_file_path,
                provide_file=fake_file)

        system_under_test.config.spungfile_path = fake_file_path
        system_under_test.foo()
        assert_correctly_frobnicated(fake_file)

With a supported standard library API for this – ‘tempfile.makepath’
for example – the generation of the filesystem path would change from
four separate function calls::

    names = tempfile._get_candidate_names()
    fake_file_path = os.path.join(tempfile.gettempdir(), names.next())

to a simple function call::

    fake_file_path = tempfile.makepath()

and have the benefit of not reaching in to a private API.

I have read the thread on Python-list (http://comments.gmane.org/gmane.comp.python.general/789339) and still don't understand the purpose of your idea.

If you need unique filesystem paths without any access to real filesystem, you can use simple counter. If you need unpredictable -- use random generator. For uniqueness use a set of already generated name. If you have additional requirements -- it is easy to modify you code for your needs.

> I have read the thread on Python-list

Thank you, and thanks for linking to that discussion.

> and still don't understand the purpose of your idea.

The purpose is to get a public API for making temporary filesystem paths with the same properties as are produced by ‘tempfile’, specifically as produced by a ‘tempfile._get_candidate_names’ generator.

The paths will be used in unit tests and will never touch the filesystem.

Some have suggested using ‘uuid’ or other random-generator APIs. Using an API that isn't explicitly designed to produce valid filesystem paths is both risky (the algorithm may not exactly produce the right behaviour), and confusing (a reader seeing use of an unrelated API will not find it obvious why that API was used for generating a filesystem path).

The unit tests need paths with properties that are already implemented in ‘tempfile’, so it seems unreasonable to suggest that functionality should be re-implemented elsewhere since it is already in the standard library.

The request is for a “get the next path generated by a ‘tempfile._get_candidate_names’ generator”, with an approved and documented public API. One suggested name is ‘tempfile.makepath’.

> The request is for a “get the next path generated by a
> ‘tempfile._get_candidate_names’ generator”, with an approved and
> documented public API.

I don't see any problem with this.  Patches welcome.

I see a problem. tempfile._get_candidate_names is implementation detail, and exposing it adds a burden for maintainers. It also adds cognitive burden for users of the tempfile module. And the idea itself looks doubtful and contradictory with the good practice of using the tempfile module.

I'm -1 for adding such function to the tempfile module.

I don't see the problem.  Every Python platform that has `mktemp` has an implementation to generate the names, and that implementation can become the public face instead of `mktemp`.  So no more of a burden than `mktemp` is; likewise for the "cognitive burden".

As far as appropriate location -- where else would you put it?

mktemp is deprecated, and I think we will get rid of it when 2.7 will be out of use. Do you want to add new deprecated from the start function?

> As far as appropriate location -- where else would you put it?

May be in third-party module on PyPI? It doesn't look as highly demanded function.

Serhiy Storchaka

> mktemp is deprecated, and I think we will get rid of it when 2.7 will be out of use.

That's fine. This is not a request to retain ‘tempfile.mktemp’.

I confused matters in my initial message, so your confusion is understandable on this point. To repeat:

The request is for a “get the next path generated by a ‘tempfile._get_candidate_names’ generator”, with an approved and documented public API. One suggested name is ‘tempfile.makepath’.


> Do you want to add new deprecated from the start function?

No. I'm asking for already-implemented and currently-maintained (i.e. not deprecated) standard library code, that is currently neither public nor documented, to gain a public API.


Ethan Furman:

> I don't see any problem with this.  Patches welcome.

Thank you, I'll work on a patch soon.

As requested by Ben Finney[1], adding my use-case here.  I'm attempting to make a hard-link from "a" to "b", but if "b" exists, os.link() will fail with an EEXISTS.  I don't want to do

 os.unlink("b")
 # power-outage here means "b" is gone
 os.link("a", "b")

I can do something like

  temp_name = tempfile.mktemp(dir=".")
  os.link("a", temp_name)
  try:
    os.rename(temp_name, "b") # docs guarantee this is atomic
  except OSError:
    os.unlink(temp_name)

but mktemp() is marked as deprecated.

I'm okay with the stray temp-file floating around to clean up in the event of power-loss after the os.link() but before the os.unlink() call, as is new info, not disposing of existing file-names



[1] https://mail.python.org/pipermail/python-list/2017-April/721641.html

You are depending on a non-portable feature of os.rename there, so I'm not convinced this makes a good use case for the Python stdlib.

On 01-May-2017, R. David Murray wrote:
> You are depending on a non-portable feature of os.rename there

What's the non-portable dependency?

If you mean the expectation that ‘os.rename’ will be atomic on
success: the documentation promises “If successful, the renaming will
be an atomic operation”, without any non-compatibility caveat
<URL:https://docs.python.org/3/library/os.html#os.rename>.

If you mean the ‘except OSError’ clause: that's not a dependency of
this example. It is handling a case that is documented to occur on
some platforms. On platforms that don't have that behaviour, the
clause is not a dependency of this use case.

Neither of those is germane to the use case being demonstrated: the
need for a temporary filesystem path as generated by ‘tempfile.mktemp’.

> so I'm not convinced this makes a good use case for the Python
> stdlib.

Note that Tim Chase is *not* presenting a use case for ‘os.rename’,
but rather a use case for ‘tempfile.mktemp’. For that purpose it seems
a valid use case.

Yes, and I'm saying his example doesn't work on Windows (on windows, it does not accomplish his goal).  So I'm not sure it is a use case appropriate for the standard library.  I'm not saying it definitely isn't, I'm just raising a doubt.

For Windows, use os.replace instead of os.rename, which will replace the destination file if it exists. However, a request to replace an existing file will be denied access if the file is currently open. Unlinking an open file isn't allowed, even if it's open with delete sharing.

I do have a workaround thanks to Gregory Ewing
https://mail.python.org/pipermail/python-list/2017-May/721649.html
which suffices for my particular use-case (generating a link to a file with a unique filename).

As my use-case is predominantly for *nix environments, the lack of Windows' os.rename() atomicity is merely a would-be-nice-for-other-people-if-it-was-available. But I'll update my documentation to reflect the edge-case. Thanks, Eryk.

>  temp_name = tempfile.mktemp(dir=".")
>  os.link("a", temp_name)

There is a race condition between generating file name and using it. tempfile.mktemp() is not much more useful that just a function that generates some names which unlikely matches the names of existing files the directory. In any case you should catch an error and repeat an attempt with different name. How much attempts to do and what additional checks to do is an application specific.

On 01-May-2017, Serhiy Storchaka wrote:

> tempfile.mktemp() is not much more useful that just a function that
> generates some names which unlikely matches the names of existing
> files the directory.

Yes. That is already useful enough to be in the standard library, and
it is there.

It is also maintained and implemented in one place, behind an
unpublished API (by using ‘next(tempfile._get_candidate_names())’).

Don't get distracted by the way ‘tempfile.mktemp’ creates files;
that's irrelevant to this issue. None of the use cases presented here
care at all about the file created, and never use that file. They
*only* want the name, generated by that standard-library algorithm.

The proposal in this issue is to have a public standard library API,
which I'm calling ‘tempfile.makepath’, to access that existing
maintained standard-library functionality.

> the lack of Windows' os.rename() atomicity

rename/replace is two system calls: NtOpenFile to get a handle for the source file and NtSetInformationFile to rename it. The difference is only that replace() sets the ReplaceIfExists field in the FileRenameInformation. The rename operation should be atomic.

If the source file is already open without delete/rename sharing, opening it to rename it will fail with sharing violation. After the handle is open, another thread could set the delete disposition on the source file (i.e. flag it to be unlinked), which will cause the rename to fail with access denied. The relative timing of the threads is the difference between getting a FileNotFoundError versus a PermissionError. The other thread could also open a handle to the destination file at this point, which also causes the rename to fail with access denied, but that's no different from what would happen if it were already open.

> The proposal in this issue is to have a public standard library API,
> which I'm calling ‘tempfile.makepath’,

It's a few years later, and tempfile.mktemp() still exists and works without raising a deprecation warning. Of course it's still marked as deprecated in the docs. Does anyone still want this tempfile.makepath() function? Or can this issue be closed?

> tempfile.mktemp() still exists and works without raising a deprecation warning. Of course it's still marked as deprecated in the docs.

Right. So, the issue is not resolved: Functionality to create a temporary file path is maintained in the standard library (good) but the public API for it is unsupported.

> Does anyone still want this tempfile.makepath() function?

That's a proposed name. But yes, a supported function in the `tempfile` standard library module, which does what's described in this issue.

> Or can this issue be closed?

The issue remains unresolved. Unless you can point us to something new which resolves this?

So no, while unresolved, this bug report should not be closed.

> So no, while unresolved, this bug report should not be closed.

The implied question was whether you or the core devs on the nosy list, upon further consideration, wanted to resolve the issue by rejecting the request, but it had simply been forgotten about for 4 years.

This function would increase security risks. Even if it will be documented as "This function is only purposed to generate file paths which looks similar to paths of temporary files generated by other functions in this module, but should not be used for any access to filesystem", it would be used to generate file paths and opening files with these paths by users which do not read documentation and do not know the right way (because they do not read documentation).

It has flaws which require to deprecate it from start.

If you want to shoot in your foot, just use private generator function tempfile._get_candidate_names(). But it should not be encouraged.

I am for closing this issue by rejecting the proposition.

> So no, while unresolved, this bug report should not be closed.

This is not a bug report, it's a feature request, and there is enough resistance in the discussion to reject it (according to the Status Quo Wins a Stalemate principle).

What may be unresolved is the problem you were trying to solve with the new feature you requested. Rather than arguing for this particular solution, I would suggest that you start a discussion about the problem and possible solutions (the python-ideas list is one place where you could bring it up).