New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
io.FileIO and io.open should support openat #57006
Comments
Right now it is painful to integrate openat() with the normal IO classes. You have to figure out the low-level flags yourself (i.e. replicate the logic and error handling from the FileIO constructor), then replicate the open() logic yourself (because you want to set the name attribute on the FileIO object before wrapping it). Therefore it would be nice if the FileIO constructor and the open() function supported openat natively. I see two possibilities:
|
A third idea is to find a way to override the low-level open() function (the one that returns a fd). openat() seems to exist only on Linux, so I'm -1 on adding new parameters to support this function only. |
Why not. It would e.g. allow to use CreateFile under Windows (the hg
openat() is POSIX: |
I prefer this suggestion. I didn't know openat(). Antoine told me that it can be used, for example, to fix security vulnerabilities like bpo-4489. |
I believe openat is new to POSIX (mandatory as of POSIX 2008). For example, it's not currently in OS X and apparently was first added to FreeBSD in 8.0. So it would have to be checked by configure and documented as platform-dependent. |
We already have os.openat: This request is to make it easier to use with the high-level IO classes. |
|
I prefer a new parameter either at the end of the arglist or possibly keyword only. The idea for both variations is to let typical users ignore the option, which would be hard to do if it is part of the prime parameter. The idea for keyword only is that we might want to add other rarely used but useful options. They have no natural order, and having say, 8 positional params is pretty wretched. (I have worked with such APIs.) |
Attached is a patch which adds dirfd= as a keyword argument. |
Thanks. Although, on second thought, I'm not sure whether Amaury's idea (allowing a custom opener) is not better... Thoughts? |
I guess that would make it more general... I'll play around with it for a bit. It mustn't become too hard to use though since the original point was to simplify the opening of files :-) |
What would you envisage the API for the custom opener to look like? |
The same as os.open(), I would say. |
Before I implement it properly, is this the kind of api that's desired? """ class MyOpener:
def __init__(self, dirname):
self.dirfd = os.open(dirname, os.O_RDONLY)
def open(self, path, flags, mode):
return os.openat(self.dirfd, path, flags, mode)
myop = MyOpener("/tmp")
f = open("testfile", "w", opener=myop.open)
f.write("hello")
""" |
Yes, although I think most people would use a closure instead of a dedicated class. |
The attached patch adds the opener keyword + tests. |
Here is my quick review:
Thank you! |
Also, the documentation should indicate what exactly is supposed to be returned by "opener". |
Updated patch:
I don't think that mode should be passed in since it is not specified in the parameters to open() (and always defaults to 0o666 anyway). Specifying the file mode should be left to the opener if needed. |
This looks good to me. You just need to add a "versionchanged" attribute |
New changeset 0d64d9ac2b78 by Ross Lagerwall in branch 'default': |
Is "an open file descriptor" correct in English? I'd have written "an opened file descriptor" instead (in 5 places). |
"open" is correct. |
Yes, 'open' is an adjective as well as a verb, and the correct one in |
Thanks (and for the English lesson ;-) ) |
See bpo-13424 for a doc request about this. |
But... os.openat() is still missing... why status is closed() ?! |
Марк, os.open added dir_fd support in 3.3, which is implemented on POSIX systems by calling openat. The dir_fd parameter is available for many os functions. This is discussed in section 1.5, Files and Directories 1. It would be nice if we could support dir_fd on Windows as well, but we'd have to bypass the CRT and Windows API to use the native NT API instead, such as NtCreateFile 2. The kernel has supported opening a file relative to a directory handle since it was release in 1993 (NT 3.1). All named kernel objects are referenced using an OBJECT_ATTRIBUTES 3 data structure. ObjectName -- a path with up to 32768 UTF-16 characters -- is relative to the RootDirectory handle if non-NULL. This is how paths relative to the process working directory are implemented, but changing the working directory isn't thread safe. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: