classification
Title: add opendir() for POSIX platforms
Type: enhancement Stage:
Components: Extension Modules Versions: Python 3.3
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: larry, pitrou, rosslagerwall, vstinner
Priority: normal Keywords:

Created on 2011-09-05 15:11 by larry, last changed 2011-09-05 16:32 by vstinner. This issue is now closed.

Messages (9)
msg143522 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2011-09-05 15:11
With the recent spate of POSIX *at() functions added to os, we now have a bunch of places in the API that take directory fds.  But afaict there's no way to get a directory fd in Python!  The only calls to opendir() in the tree are internal, in os.listdir() and in the import machinery.  (Though in practice most people will use AT_FDCWD anyway.)

I propose adding a new function, os.opendir(), the implementation to be much the same as (aka a hacked-up copy and paste of) os.unlink() in Modules/posixmodule.c.  I'd be happy to contribute the patch.
msg143525 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-09-05 15:14
opendir opens a C dirent structure with an underlying file descriptor.

However, to open a directory file descriptor, simple use:
 os.open("/tmp", os.O_RDONLY)

This can then be used as the fd to the functions which require a directory fd like os.openat()

Cheers
Ross
msg143527 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-09-05 15:18
Also see fdopendir(3) which allows you to pass an open file descriptor to get a C dirent struct.

This is implemented in the os module too but instead of returning a struct, it returns a list.
msg143536 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2011-09-05 15:53
Well, there's no os.fdopendir(); I think you're referring to fdlistdir(), which uses the C function fdopendir() internally.  The DIR structure is not exposed to the Python caller at any point.

I did miss the whole opendir-returns-a-DIR-not-a-fd thing, hopefully that's my dumb thing of the day.  This suggests opendir() would have to call dirfd() on your behalf.

Would it be safe to call close() on the fd returned from dirfd(opendir())?  If not, I guess we'd need a special closedir() function, which would use the C functions fdopendir() then closedir() internally.

... or we could forget the whole thing, I guess.
msg143537 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-09-05 15:58
Well, is there any case where fdlistdir() is not sufficient?
msg143538 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2011-09-05 16:00
fdlistdir() is largely irrelevant to the discussion.  I was proposing adding a function to open directory fds, because there isn't one; fdlistdir(), like many other POSIX functions available in Python, consumes directory fds.
msg143545 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-09-05 16:16
> I was proposing adding a function to open directory fds, because 
> there isn't one; fdlistdir(), like many other POSIX functions 
> available in Python, consumes directory fds.

I don't think I understand. This already works:

>>> fd = os.open("Misc", os.O_RDONLY)
>>> os.fdlistdir(fd)
['python.pc', 'README.coverity', 'ACKS.orig', 'NEWS.orig', 'HISTORY', 'python.man', 'valgrind-python.supp', 'TextMate', 'python-config.in', 'NEWS', 'RPM', 'vgrindefs', 'svnmap.txt', 'python-wing4.wpr', 'gdbinit', 'build.sh', 'README.AIX', 'Vim', 'SpecialBuilds.txt', 'indent.pro', 'python-wing3.wpr', 'ACKS', 'README', 'README.valgrind', 'Porting', 'python.pc.in']

Is there anything you want to do on a "directory fd" except listing its contents?
msg143546 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2011-09-05 16:22
> Is there anything you want to do on a "directory fd"
> except listing its contents?

In the first message in this bug, I wrote: "With the recent spate of POSIX *at() functions added to os, we now have a bunch of places in the API that take directory fds."  To be specific:  faccessat, fchmodat, fchownat, fstatat, futimesat, linkat, mkdirat, mknodat, openat, readlinkat, renameat, symlinkat, unlinkat, utimensat, mkfifoat.

At the time I created this ticket I didn't realize you could just call open() on a directory.  It seem that works fine and is supported everywhere that these *at functions exist, so perhaps it's best if we just close this ticket.
msg143549 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-09-05 16:32
> At the time I created this ticket I didn't realize you could
> just call open() on a directory.

Yes, os.open or os.openat.
History
Date User Action Args
2011-09-05 16:32:07vstinnersetnosy: + vstinner
messages: + msg143549
2011-09-05 16:31:16vstinnersetstatus: open -> closed
resolution: works for me
2011-09-05 16:22:20larrysetmessages: + msg143546
2011-09-05 16:16:26pitrousetmessages: + msg143545
2011-09-05 16:00:45larrysetmessages: + msg143538
2011-09-05 15:58:17pitrousetnosy: + pitrou
messages: + msg143537
2011-09-05 15:53:36larrysetmessages: + msg143536
2011-09-05 15:18:18rosslagerwallsetmessages: + msg143527
2011-09-05 15:14:59rosslagerwallsetnosy: + rosslagerwall
messages: + msg143525
2011-09-05 15:11:27larrycreate