classification
Title: Disable size checks in mmap.mmap()
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: haypo, neologix, rosslagerwall, schmir, superbobry
Priority: normal Keywords:

Created on 2011-07-14 13:20 by superbobry, last changed 2011-07-22 05:16 by superbobry. This issue is now closed.

Messages (8)
msg140330 - (view) Author: Sergei Lebedev (superbobry) Date: 2011-07-14 13:20
Current `mmap` implementation raises a ValueError if a sum of offset and length exceeds file size, as reported by `fstat`. While perfectly valid for most use-cases, it doesn't work for "special" files, for example:

>>> with open("/proc/sys/debug/exception-trace", "r+b") as f:
...     mmap.mmap(f.fileno(), 0)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: mmap offset is greater than file size

Same goes for almost any other /proc file, because most of them have S_ISREG() == True and st_size = 0.

How about adding a keyword argument to `mmap.mmap()`, which disables fstat-based size checks?
msg140447 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-07-15 17:34
A couple remarks:
- if st_size == 0, then you have to specify an explicit length (your example passes 0, which would fail with EINVAL without the check)
- most enties in /proc don't support mmap file operation, so it's likely to fail anyway (with ENODEV, EACCESS...). Do you have an example of a /proc entry with st_size == 0 that can be mmapped (mapping /proc/sys/debug/exception-trace fails with EACCESS)?

> How about adding a keyword argument to `mmap.mmap()`, which disables
> fstat-based size checks?

I don't like the idea of adding an argument which doesn't have a counterpart in the POSIX version (especially to address such corner cases).
msg140805 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-07-21 10:42
> I don't like the idea of adding an argument which doesn't have a
> counterpart in the POSIX version (especially to address such corner
> cases).

Indeed, it seems rather messy for a corner case that may well not exist.

If there are definite cases where this usage is needed, a solution may then have to be considered.
msg140806 - (view) Author: Sergei Lebedev (superbobry) Date: 2011-07-21 11:02
> Do you have an example of a /proc entry with st_size == 0 that can be mmapped 
> (mapping /proc/sys/debug/exception-trace fails with EACCESS)?
Yes, I've ran into the issue, while trying to mmap /proc/xen/xsd_kva, which is an 
interface to XenBus [1]. Unfortunately, I'm not aware of any other cases. 

By the way, I've checked mmap(2) manpage -- it looks like the C-version has nothing 
against mmaping 0-sized files, Why does Python's `mmap` still checks file size?

[1] http://wiki.xensource.com/xenwiki/XenBus
msg140824 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-07-21 19:05
> By the way, I've checked mmap(2) manpage -- it looks like the C-version has
> nothing
> against mmaping 0-sized files, Why does Python's `mmap` still checks file
> size?

It doesn't check explicitely that the size is non-0, but rather that
the offset is is less than the file size.
Passing mmap(2) a 0 length doesn't make much sense anyway - for
example, Linux returns EINVAL:
http://lxr.free-electrons.com/source/mm/mmap.c?a=avr32#L979
"""
unsigned long do_mmap_pgoff(...)
[...]
        if (!len)
                 return -EINVAL;
"""

>> Do you have an example of a /proc entry with st_size == 0 that can be
>> mmapped
>> (mapping /proc/sys/debug/exception-trace fails with EACCESS)?
> Yes, I've ran into the issue, while trying to mmap /proc/xen/xsd_kva, which
> is an
> interface to XenBus [1]. Unfortunately, I'm not aware of any other cases.

That's what I thought, it's really uncommon: in that case, I'm
reluctant to making such a change, for the reason explained above.
Ross, Victor?
msg140834 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-21 23:13
> That's what I thought, it's really uncommon: in that case, I'm
> reluctant to making such a change, for the reason explained above.
> Ross, Victor?

Why do you want a mmap? Why not using a file?
msg140861 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-07-22 05:11
> That's what I thought, it's really uncommon: in that case, I'm
> reluctant to making such a change, for the reason explained above.
> Ross, Victor?

Even if Linux did accept a a length of 0 (which it doesn't since 2.6.12), what would be the point? Wouldn't you be mapping *nothing*?

Yeah, I'm against this change.
msg140862 - (view) Author: Sergei Lebedev (superbobry) Date: 2011-07-22 05:16
> Passing mmap(2) a 0 length doesn't make much sense anyway - for
example, Linux returns EINVAL.
Yup, but passing mmap(2) a zero-sized file and a non-zero length works just fine.

> Why do you want a mmap? Why not using a file?
Because Xen requires me to mmap it. Anyway, you're right, the use-case is too uncommon.
History
Date User Action Args
2011-07-22 05:16:16superbobrysetstatus: open -> closed
resolution: rejected
messages: + msg140862
2011-07-22 05:11:28rosslagerwallsetmessages: + msg140861
2011-07-21 23:16:23schmirsetnosy: + schmir
2011-07-21 23:13:34hayposetmessages: + msg140834
2011-07-21 19:05:08neologixsetmessages: + msg140824
2011-07-21 11:54:21hayposetnosy: + haypo
2011-07-21 11:02:52superbobrysetmessages: + msg140806
2011-07-21 10:42:49rosslagerwallsetmessages: + msg140805
2011-07-15 17:34:13neologixsetmessages: + msg140447
2011-07-14 15:08:47pitrousetnosy: + neologix, rosslagerwall

versions: + Python 3.3, - Python 2.6
2011-07-14 13:20:59superbobrycreate