classification
Title: mailbox.Maildir doesn't create subdir structure when create=True and base dir exists
Type: behavior Stage: resolved
Components: email, Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: barry, louielu, miss-islington, r.david.murray, serhiy.storchaka, webknjaz
Priority: normal Keywords: patch

Created on 2017-04-17 07:47 by webknjaz, last changed 2019-07-13 14:59 by miss-islington. This issue is now closed.

Files
File name Uploaded Description Edit
poc.py louielu, 2017-04-17 09:26 PoC of the current Maildir hehavior
Pull Requests
URL Status Linked Edit
PR 1163 merged webknjaz, 2017-04-17 07:47
PR 14749 merged miss-islington, 2019-07-13 14:47
PR 14750 merged miss-islington, 2019-07-13 14:47
Messages (10)
msg291789 - (view) Author: Sviatoslav Sydorenko (webknjaz) * Date: 2017-04-17 07:47
Hi,

I've faced an issue w/ `mailbox.Maildir()`. The case is following:
1. I create a folder with `tempfile.TemporaryDirectory()`, so it's empty
2. I pass that folder path as an argument when instantiating `mailbox.Maildir()`
3. Then I receive an exception happening because "there's no such file or directory" (namely `cur`, `tmp` or `new`) during interaction with Maildir

**Expected result:** subdirs are created during `Maildir()` instance creation.

**Actual result:** subdirs are assumed as existing which leads to exceptions during use.

**Workaround:** remove the actual dir before passing the path to `Maildir()`. It will be created automatically with all subdirs needed.

**Fix:** PR linked. Basically it adds creation of subdirs regardless of whether the base dir existed before.
msg291793 - (view) Author: Louie Lu (louielu) * Date: 2017-04-17 09:26
I think this patch make the behavior changed.

Documentation wrote that: "If create is True, the mailbox is created if it does not exist.", the current version did that exactly, it won't create subdir (tmp, new, and cur) when dir exists.

The situation face here is that TemporaryDirectory() create a valid dir without Maildir structure subdir (tmp, new, and cur), and the `create` parameter won't create subdir (because the dir is exist!).

Not sure if this behavior change is worth or not. I'll also upload an PoC for this.
msg291794 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-17 10:30
Just create a subdirectory inside the tempdir to hold your Maildir folder.  I think it is not worth complicating the API for this use case, since it does have a simple solution.
msg291795 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-17 11:12
I concur with David. Rather than using a temporary directory as a Maildir folder:

    with tempfile.TemporaryDirectory() as tmpdir:
        with mailbox.Maildir(tmpdir, create=True) as box:
            ...

create a Maildir folder in a temporary directory:

    with tempfile.TemporaryDirectory() as tmpdir:
        with mailbox.Maildir(os.path.join(tmpdir, 'mail'), create=True) as box:
            ...
msg291810 - (view) Author: Sviatoslav Sydorenko (webknjaz) * Date: 2017-04-17 19:45
> create a Maildir folder in a temporary directory:
> 
>     with tempfile.TemporaryDirectory() as tmpdir:
>         with mailbox.Maildir(os.path.join(tmpdir, 'mail'), create=True) as box:
>             ...

Yeah, I came up with the same solution. It just seems weird to me.

The doc says "If create is True, the mailbox is created if it does not exist.". So it operates the term mailbox, not a folder.

I would assume that a valid mailbox [1] is the whole **valid** directory structure with subfolders. If the corresponding subdirs are missing the folder should not be considered as a valid maildir, right?

Given above, I think the documentation doesn't clearly promise the same behavior that the code does. Shouldn't this be fixed? (Either docs, or code)

[1] http://www.qmail.org/man/man5/maildir.html
msg291811 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-17 19:58
Right.  Which is why you get an error if you try to use an empty directory as if it was a maildir :)

create=True creates the *directory* and initializes it.  That seems clear to me, and exactly what I would expect.  "The mailbox" is the directory.  If it already exists, it should be a Maildir folder.  I wouldn't want a program messing with an existing directory if it was invalid...it might trash my data :)

I don't think even a doc change is needed here, but if you want to propose something we can evaluate it.
msg347808 - (view) Author: Sviatoslav Sydorenko (webknjaz) * Date: 2019-07-13 12:46
Hi, I've tried clarifying this in docs as suggested @ https://github.com/python/cpython/pull/1163
msg347834 - (view) Author: miss-islington (miss-islington) Date: 2019-07-13 14:47
New changeset e44184749c2fd0921867ea5cd20b8e226c2146c2 by Miss Islington (bot) (Sviatoslav Sydorenko) in branch 'master':
bpo-30088: Document that existing dir structure isn't verified by mailbox.Maildir (GH-1163)
https://github.com/python/cpython/commit/e44184749c2fd0921867ea5cd20b8e226c2146c2
msg347837 - (view) Author: miss-islington (miss-islington) Date: 2019-07-13 14:59
New changeset d1524148cd08f00c0b7c1dfdf698bf96c246350d by Miss Islington (bot) in branch '3.7':
bpo-30088: Document that existing dir structure isn't verified by mailbox.Maildir (GH-1163)
https://github.com/python/cpython/commit/d1524148cd08f00c0b7c1dfdf698bf96c246350d
msg347838 - (view) Author: miss-islington (miss-islington) Date: 2019-07-13 14:59
New changeset b815669c833c543b0f6696c3121a179f6b2383a6 by Miss Islington (bot) in branch '3.8':
bpo-30088: Document that existing dir structure isn't verified by mailbox.Maildir (GH-1163)
https://github.com/python/cpython/commit/b815669c833c543b0f6696c3121a179f6b2383a6
History
Date User Action Args
2019-07-13 14:59:52miss-islingtonsetmessages: + msg347838
2019-07-13 14:59:35miss-islingtonsetmessages: + msg347837
2019-07-13 14:49:08asvetlovsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.8, Python 3.9
2019-07-13 14:47:32miss-islingtonsetpull_requests: + pull_request14545
2019-07-13 14:47:26miss-islingtonsetkeywords: + patch
pull_requests: + pull_request14544
2019-07-13 14:47:17miss-islingtonsetnosy: + miss-islington
messages: + msg347834
2019-07-13 12:46:20webknjazsetmessages: + msg347808
2017-04-17 19:58:17r.david.murraysetmessages: + msg291811
2017-04-17 19:45:47webknjazsetmessages: + msg291810
2017-04-17 11:12:28serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg291795
2017-04-17 10:30:35r.david.murraysetmessages: + msg291794
versions: - Python 2.7, Python 3.5, Python 3.6
2017-04-17 09:26:11louielusetfiles: + poc.py
nosy: + louielu
messages: + msg291793

2017-04-17 08:11:55serhiy.storchakasetnosy: + barry, r.david.murray
stage: patch review

components: + email
versions: - Python 3.3, Python 3.4
2017-04-17 07:47:54webknjazcreate