classification
Title: Use Linux O_TMPFILE flag in tempfile.TemporaryFile?
Type: Stage:
Components: Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, georg.brandl, josh.r, ncoghlan, neologix, pitrou, python-dev, serhiy.storchaka, socketpair, vstinner
Priority: normal Keywords: patch

Created on 2014-05-16 09:17 by vstinner, last changed 2015-10-20 22:20 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
tempfile_o_tmpfile.patch vstinner, 2014-05-16 09:17 review
tempfile_o_tmpfile3.patch vstinner, 2014-05-18 10:01 review
tempfile_comment.patch vstinner, 2015-10-20 19:39 review
Messages (27)
msg218648 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-16 09:17
Linux 3.11 introduced a new file flag "O_TMPFILE". The flag is exposed in Python, see the issue #18673.

"O_TMPFILE is a new open(2)/openat(2) flag that makes easier the creation of secure temporary files. Files opened with the O_TMPFILE flag are created but they are not visible in the filesystem. And as soon as they are closed, they get deleted - just as a file you would have opened and unlinked."
http://kernelnewbies.org/Linux_3.11#head-8be09d59438b31c2a724547838f234cb33c40357

Does it make sense to use this flag in tempfile.TemporaryFile?

Attached patch is a work-in-progress patch for tempfile.

> if hasattr(_os, 'O_TMPFILE'):
>     _O_TMPFILE = _os.O_TMPFILE
> elif _sys.platform == 'linux':
>     __O_TMPFILE = 0o20000000
>     _O_TMPFILE = (__O_TMPFILE | _os.O_DIRECTORY)

The second if should be removed. I used it because my Linux kernel (3.14) supports the flag, but the constant is not defined yet in C headers of my C library (glibc 2.18).

> flags = (flags | _O_TMPFILE) & ~_os.O_CREAT

O_CREAT is incompatible with O_TMPFILE.

Bonus point of the flag: no need to compute a random name! Just pass the temporary directory.

To do: test the patch on Linux < 3.11 to see how the flag is interpreted. If the flag is ignored, we open the directory in write mode! That's insafe. If the flag raises an error, we should fallback to the current implementation and remember that the flag is not supported.

I implemented something similar for O_CLOEXEC and SOCK_CLOEXEC flags (PEP 433).
msg218649 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-16 09:34
I don't think we can use this by default, or it will break the expected semantics of temporary files under Unix (visible by other processes).
msg218650 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-16 09:47
"I don't think we can use this by default, or it will break the expected semantics of temporary files under Unix (visible by other processes)."

I proposed to change TemporaryFile, not NamedTemporaryFile. Do you mean that other processes are supposed to have access to the temporary file descriptor? Access through /proc/pid/fd/<tmp_fd>?

O_TMPFILE should increase the security because there is no more race condition between os.open() and os.unlink() (window where an attack can access the file).

My patch uses O_EXCL. It makes possible to use linkat() to create a path for the temporary file (I didn't try it, but I read that it's possible). I don't know if using O_EXCL should be the default.
msg218653 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-16 11:21
> I proposed to change TemporaryFile, not NamedTemporaryFile.

Ah, sorry. Then it sounds ok.
(I couldn't find any documentation for O_TMPFILE, though)
msg218660 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-16 12:55
It looks like O_TMPFILE is supported by tmpfs (3.11), ext3 (3.11), ext4 (3.11), XFS (3.15). It looks like BTRFS will also support the O_TMPFILE:
https://btrfs.wiki.kernel.org/index.php/Project_ideas#Implement_O_TMPFILE_support

--

It looks like os.open() fails with OSError(95, 'Operation not supported') if the filesystem of the directory doesn't support TMPFILE. In this case, a fallback to the current implementation should be enough. I don't think that we need to remember that the directory doesn't support TMPFILE. The directory may be on a different filesystem at the next call.

haypo@smithers$ ~/prog/python/default/python 
Python 3.5.0a0 (default:5e98a50e0f55, May 16 2014, 10:44:10) 
>>> import tempfile
>>> tempfile._O_TMPFILE
4259840
>>> f=tempfile.TemporaryFile(dir='.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/haypo/prog/python/default/Lib/tempfile.py", line 507, in TemporaryFile
    fd = _os.open(dir, flags, 0o600)
OSError: [Errno 95] Operation not supported: '.'

haypo@smithers$ df .
Sys. de fichiers   Taille Utilisé Dispo Uti% Monté sur
192.168.0.42:/test    96G    9,1G   83G  10% /home/haypo/nfs
msg218661 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-16 13:08
It looks like open() ignores O_TMPFILE (0o20000000) on old Linux kernels. Test on Linux 3.2:

>>> fd=os.open("/tmp", os.O_RDWR | O_TMPFILE, 0o600)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 21] Is a directory: '/tmp'

>>> fd=os.open("/tmp", os.O_RDWR | os.O_DIRECTORY, 0o600)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 21] Is a directory: '/tmp'

So we should catch OSError(21, "Is a directory: '/tmp'") and fallback to the current implementation (random name, unlink), and remember that the kernel version is too old.
msg218667 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-16 15:10
> So we should catch OSError(21, "Is a directory: '/tmp'") and fallback
> to the current implementation (random name, unlink), and remember that
> the kernel version is too old.

Just catch any OSError?
msg218670 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-16 15:20
> Just catch any OSError?

If possible, I would prefer to not retry O_TMPFILE each time if the kernel version does not support the flag.

Pseudo-code:
--
if o_tmpfile_supported:
   try:
      fd = os.open(dir, os.O_TMPFILE | ...)
   except IsADirectoryError:
      # Linux kernel older than 3.11 ignores O_TMPFILE flag
      o_tmpfile_supported = False
   except OSError:
      # the filesystem doesn't support O_TMPFILE
      pass
   else:
      return io.open(fd, ...)
   # fallback to unsafe but portable implementation

# current code generating a name and using unlink
---
msg218671 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-05-16 15:29
> > Just catch any OSError?
> 
> If possible, I would prefer to not retry O_TMPFILE each time if the kernel version does not support the flag.

How likely it is to have a glibc flag that's not supported by the kernel
(on a normal setup, not a self-compiled distro)?
msg218690 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-05-17 01:59
Reasonably common, I believe. For example, Red Hat ships a Developer
Toolset, so you may be building with an up to date gcc on RHEL 6 or 7, but
still support deploying against the older kernel in RHEL 5.
msg218738 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-18 09:52
tempfile_o_tmpfile2.patch: Updated patch to handle OS errors.

I'm not sure that __O_TMPFILE has the same value on all architectures.

The O_TMPFILE flag was added to fcntl.h in the glibc 2.19 (released the 8 Feb 2014):
https://sourceware.org/git/?p=glibc.git;a=commit;h=ffdd31816a67f48697ea4d6b852e58d2886d42ca

My Linux Fedora 20 uses the glibc 2.18.

I removed the hardcoded constant from my patch. Add this to Lib/tempfile.py to test manually if you have a glibc older than 2.19:

# after "if hasattr(_os, 'O_TMPFILE'):"
elif _sys.platform == 'linux':
    __O_TMPFILE = 0o20000000
    _O_TMPFILE = (__O_TMPFILE | _os.O_DIRECTORY)
msg218739 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-18 10:01
(Oops, I made a mistake, the hardcoded constant was still present in my patch 2.)

Patch 3 uses tempfile._O_TMPFILE_WORKS variable to check if the O_TMPFILE flag is avaialble and works.

Use "os.O_TMPFILE = 0o20000000 | os.O_DIRECTORY" to try my patch with glibc older than 2.19.
msg219792 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-05 10:19
Can someone please review  tempfile_o_tmpfile3.patch ?
msg219803 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-06-05 11:56
It would be nice if the patch added a pointer to the O_TMPFILE documentation (if that exists) and mentioned that it is Linux-specific.
Otherwise, it looks good to me.
msg219807 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-05 12:29
New changeset 4b51a992cb70 by Victor Stinner in branch 'default':
Issue #21515: tempfile.TemporaryFile now uses os.O_TMPFILE flag is available
http://hg.python.org/cpython/rev/4b51a992cb70
msg219808 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-05 12:30
> It would be nice if the patch added a pointer to the O_TMPFILE documentation (if that exists) and mentioned that it is Linux-specific.

I modified TemporaryFile documentation to mention that the O_TMPFILE
flag is used if available and if the flag "works".
msg220037 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-06-08 15:50
Minor inconsistency in Lib/tempfile.py:
                # Set flag to None to not try again.
                _O_TMPFILE_WORKS = False

s/None/False/
msg220059 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-08 22:06
New changeset 8b93cdccd872 by Victor Stinner in branch 'default':
Issue #21515: Fix typo in a comment, thanks Arfrever for the report
http://hg.python.org/cpython/rev/8b93cdccd872
msg253196 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-10-19 21:28
Suppose conditions:
- Old linux kernel ignoring flag
- malicious hacker force use of PLAIN FILE instead of directory

On new kernel it will fail
On old kernel it will just open that file!

So, we can make a HACK! Just add last slash to directory name. This will not hurt on new kernels, but protect on old kernels.

tests should also test a case when directory is symlink really.
msg253213 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-20 07:54
> Suppose conditions:
> - Old linux kernel ignoring flag
> - malicious hacker force use of PLAIN FILE instead of directory

Is it a theorical bug, or are you able to reproduce it?

Old Linux kernel ignores the 0o20000000 bit but O_TMPFILE is 0o20000000 | os.O_DIRECTORY. So the kernel still ensures that the path is a directory. tempfile.TemporaryFile() tries to open the path with:

   os.open(path, os.O_RDWR |os.O_EXCL | os.O_TMPFILE)

if the 0o20000000 bit is ignored by old kernel, it becomes:

   os.open(path, os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)

You cannot open a regular file with these flags:

>>> open('x', 'w').close()
>>> os.open('x', os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotADirectoryError: [Errno 20] Not a directory: 'x'

You cannot open a directory with these flags:

>>> os.open('.', os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IsADirectoryError: [Errno 21] Is a directory: '.'

Same behaviour for symbolic link to a regular file or to a directory.

Please open a new issue if you consider that you found a bug, but please write a short script reproducing the bug.
msg253238 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-10-20 18:02
Okay, seemes it is not documented that

os.open('.', os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)

Should return EISDIR  

I did not found that in Linux manpages. Using undocumented features is bad. Maybe I should report this to Michael Kerrisk to update manpage ?
msg253240 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-10-20 18:11
Well, it's not said explicit, that O_DIRECTORY cannot be combined with O_RDWR.

So, everything is valid now, very hacky, but works without bugs.

It will be nice, if someone comment that hacks in source code
msg253245 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-20 19:30
2015-10-20 20:02 GMT+02:00 Марк Коренберг <report@bugs.python.org>:
> Okay, seemes it is not documented that
>
> os.open('.', os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)
>
> Should return EISDIR

You cannot open a directory to write, only to read.
msg253247 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-20 19:39
> It will be nice, if someone comment that hacks in source code

I don't understand why you keep calling this a hack. It's part of open() contract, and I'm quite sure that it was a deliberate choice to declare O_TMPFILE as O_DIRECTY|new_bit. See for example this comment:
https://lwn.net/Articles/560834/

I wrote a patch to explain that it's fine to call open() with O_TMPFILE on old kernels to check if the flag is supported: see attached patch.
msg253248 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-10-20 19:56
Huge thanks for that patch. Now things are much cleaner.
msg253265 - (view) Author: Roundup Robot (python-dev) Date: 2015-10-20 22:15
New changeset dc2deecb2346 by Victor Stinner in branch '3.5':
Issue #21515: Elaborate tempfile.TemporaryFile() comment
https://hg.python.org/cpython/rev/dc2deecb2346
msg253266 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-20 22:20
> Huge thanks for that patch. Now things are much cleaner.

I understand that the patch looks good to you, so I pushed it to Python 3.5 & 3.6. I close again the issue. Thanks for your analasys of tempfile.TemporaryFile() :-)
History
Date User Action Args
2015-10-20 22:20:25vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg253266
2015-10-20 22:15:36python-devsetmessages: + msg253265
2015-10-20 19:56:00socketpairsetmessages: + msg253248
2015-10-20 19:39:50vstinnersetstatus: closed -> open
files: + tempfile_comment.patch
resolution: fixed -> (no value)
messages: + msg253247
2015-10-20 19:30:01vstinnersetmessages: + msg253245
2015-10-20 18:11:00socketpairsetmessages: + msg253240
2015-10-20 18:02:27socketpairsetmessages: + msg253238
2015-10-20 07:54:21vstinnersetmessages: + msg253213
2015-10-19 21:28:35socketpairsetnosy: + socketpair
messages: + msg253196
2014-06-10 09:19:50vstinnersetstatus: open -> closed
resolution: fixed
2014-06-08 22:06:02python-devsetmessages: + msg220059
2014-06-08 15:50:51Arfreversetnosy: + Arfrever
messages: + msg220037
2014-06-05 12:30:39vstinnersetmessages: + msg219808
2014-06-05 12:29:34python-devsetnosy: + python-dev
messages: + msg219807
2014-06-05 11:56:56pitrousetmessages: + msg219803
2014-06-05 10:19:06vstinnersetmessages: + msg219792
2014-05-26 11:02:06serhiy.storchakasetnosy: + serhiy.storchaka
2014-05-18 10:01:16vstinnersetfiles: + tempfile_o_tmpfile3.patch

messages: + msg218739
2014-05-18 09:53:21vstinnersetfiles: - tempfile_o_tmpfile2.patch
2014-05-18 09:52:48vstinnersetfiles: + tempfile_o_tmpfile2.patch

messages: + msg218738
2014-05-17 02:06:27josh.rsetnosy: + josh.r
2014-05-17 01:59:07ncoghlansetmessages: + msg218690
2014-05-16 15:29:18pitrousetmessages: + msg218671
2014-05-16 15:20:53vstinnersetmessages: + msg218670
2014-05-16 15:10:23pitrousetmessages: + msg218667
2014-05-16 13:08:03vstinnersetmessages: + msg218661
2014-05-16 12:55:51vstinnersetmessages: + msg218660
2014-05-16 11:21:10pitrousetmessages: + msg218653
2014-05-16 09:47:35vstinnersetmessages: + msg218650
2014-05-16 09:34:47pitrousetnosy: + georg.brandl, ncoghlan, pitrou
messages: + msg218649
2014-05-16 09:17:21vstinnercreate