classification
Title: Add the root_dir and dir_fd parameters in glob.glob()
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: akuchling, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2019-09-12 18:31 by serhiy.storchaka, last changed 2020-10-20 16:45 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 16075 merged serhiy.storchaka, 2019-09-12 18:35
PR 22805 merged serhiy.storchaka, 2020-10-20 05:53
Messages (6)
msg352223 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-12 18:31
The pattern for the glob() function can be relative and absolute. If it is relative, paths are searched from the current directory. If you want to search them in other directory, you need either change the current directory (which affects other parts of your program) or pass a concatenation of the escaped root directory and a pattern:

   glob.glob(os.path.join(glob.escape(root_dir), pattern))

Most code (even in the stdlib and tools) forget to escape the root directory. It works only until it does not contain metacharacters ('*?[').

When you need paths relative to the root directory, you need to "remove" the root_dir prefix from results (using os.path.relpath() at best).

The proposed PR adds two new parameters in glob.glob() and glob.iglob(): root_dir and dir_fd.

root_dir specifies the root directory for relative pattern. Its effect is the same as chdir before calling glob(). It is similar to the root_dir parameter of shutil.make_archive. For example, you can add py-files in the specified directory to the ZIP archive by:

    with zipfile.ZipFile(archive, 'w') as zf:
        for filename in glob.glob('**/*.py', recursive=True, root_dir=root_dir):
            zf.write(os.path.join(root_dir, filename), arcname=filename)

Adding to archive and copying are simpler if you have paths relative to the specified directory.

The dir_fd parameter is similar to root_dir, but it specifies the root directory as an open directory descriptor instead of a path (as in many os functions). It adds security (nobody can rename and replace the root directory in process) and performance (because of shorter paths).

root_dir and dir_fd can be combined. root_dir is relative to dir_fd and the pattern is relative to root_dir.

If root_dir is absolute, dir_fd is ignored. If the pattern is absolute, root_dir and dir_fd are ignored.
msg370545 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-06-01 09:23
After adding this feature to iglob I am going to add it to other functions which work recursively with a directory tree.

The only question: should we add two parameters root_dir and dir_fd or combine them in a single rood_dir (which can be either path or file descriptor or a special object combining a file descriptor and a relative path)? Or maybe merge one variant, test it and change it before feature freeze if needed?
msg371832 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-06-18 19:08
New changeset 8a64ceaf9856e7570cad6f5d628cce789834e019 by Serhiy Storchaka in branch 'master':
bpo-38144: Add the root_dir and dir_fd parameters in glob.glob(). (GH-16075)
https://github.com/python/cpython/commit/8a64ceaf9856e7570cad6f5d628cce789834e019
msg378993 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2020-10-19 19:17
I'm digging around in the sys.audit() feature.  Did the sys.audit("glob.glob", ...) call disappear in commit 8a64ceaf9856e7570cad6f5d628cce789834e019, or is something clever going on that I missed?
msg379089 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-20 05:54
Thank you for catching this. It was an error introduced by merge.
msg379143 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-20 16:45
New changeset 1d3469988e2c1f53ca84ffdc979d548c04ba3906 by Serhiy Storchaka in branch 'master':
bpo-38144: Re-add accidentally removed audition for glob. (GH-22805)
https://github.com/python/cpython/commit/1d3469988e2c1f53ca84ffdc979d548c04ba3906
History
Date User Action Args
2020-10-20 16:45:46serhiy.storchakasetmessages: + msg379143
2020-10-20 05:54:59serhiy.storchakasetmessages: + msg379089
2020-10-20 05:53:07serhiy.storchakasetpull_requests: + pull_request21762
2020-10-19 19:17:14akuchlingsetnosy: + akuchling
messages: + msg378993
2020-06-18 19:08:39serhiy.storchakasetmessages: + msg371832
2020-06-01 09:23:07serhiy.storchakasetmessages: + msg370545
versions: + Python 3.10, - Python 3.9
2019-09-12 18:35:09serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request15696
2019-09-12 18:31:01serhiy.storchakacreate