This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: email module should have a way to prepend and insert headers
Type: enhancement Stage:
Components: email, Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, kbandla, r.david.murray, suryak
Priority: normal Keywords: patch

Created on 2013-06-05 04:35 by kbandla, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
suryak.patch suryak, 2014-03-03 15:03 insert_header() to insert header at the beginning review
Messages (10)
msg190641 - (view) Author: Kiran Bandla (kbandla) Date: 2013-06-05 04:35
This issue is in message.py in the email module. The add_header method should insert the new header on the top of the existing headers. The current function implementation appends the header at the end of the existing headers.
msg190658 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-06-05 13:21
This is by design.  I agree, however, that there should be a way to do this when an application needs it, so I'm changing this to a feature request.
msg190661 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-06-05 14:07
In a somewhat similar vein, replace_header() retains the original header order.  Not quite what the OP wants, but useful.

The problem I had originally with a position-aware method is getting the API right.  I didn't want to add a position argument to add_header() (and still don't).  So I think a position-aware API would have to be a different method, perhaps with an API somewhat like list.insert()?  It doesn't look like OrderedDict provides any guidance here.
msg190662 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-06-05 14:26
My preliminary thought (and I haven't checked the code yet to make sure this will actually work), is that under the new email policies we are dealing with full-blown header objects, meaning that they can know the header name they represent.  This should theoretically make it possible to have a 'create header' function to create a header object (there may be one already...it's been way too long since I've worked on this code) and then have:

  message[:0] = [my_new_header]

do the right thing.  As well as all the other list methods working as expected.  We'd also want a 'find_header' method that takes the header name and returns the index of the header in the list.

But, as I say, this is just a preliminary thought, it needs careful consideration before we decide to actually do something.
msg212642 - (view) Author: Surya K (suryak) Date: 2014-03-03 15:03
I've done the following changes:

1. create insert_header(): Which places header at the beginning.
For that I created set() method which actually takes an argument `pos` default to 0.

2. __setitem__() uses set() (new method) method while fixing `pos` to end.
3. _parse_values(), new method created to not duplicate the code in insert_header() and add_header() since both of them merely do same job.

Please let me know your opinions on it.
msg212645 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-03 15:42
The patch is a reasonable effort, thanks.  However, before we can really evaluate any code, we need to agree on an API.

The Message object presents a hybrid dictionary/sequence API over the headers.  In the Sequence API, we insert something into the list using 'insert(pos, object)`.  dict has no 'insert' method, so we don't need to worry about an API clash there.

So we could define the method insert_header to have the following signature:

  insert_header(pos, name, value)

add_header is still required, because we can't use insert_header to append.  (It would then be nice if it were named append_header...but ignore that for now).

However, there is almost no other context in which one interacts with the header list via a header's index in the list of headers.  Message does not support the 'index' method.

An alternate API, then, might be something like:

    insert_header_after(existing_name, new_name, value)

This would be analogous to replace_header.

The trouble with this, and the trouble with defining a header_index, is that multiple headers can have the same name.  Message's answer to this currently is to have both a 'get' method and a 'get_all' method.  The former returns the first match, the latter all of them.

The reason this matters, by the way, is that one of the motivations for insert_header in my mind is the ability to create a sensible flow of headers: have the routing and forwarding headers (received, resent-xxx, etc) headers be before the From/to/date, etc headers.  But to do that, you need to be able to insert them *after* the other headers with the same name.

You could make 'insert after last instance of name' the default behavior of insert_header_after, but then we sill don't have a way to insert a header in an arbitrary location.

Another possibility is 'insert_header' combined with both 'header_index' and 'header_index_all'.

I *think* I'm leaning toward the latter (with my biggest hesitation being that insert_header is pretty much the only place you can use the index information returned by the index methods), but this kind of API design issue is something we should run by the email-sig mailing list.  Feel free to post something there yourself, otherwise I will do so after I finish the 'whatsnew' work.
msg212646 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-03-03 15:46
(Aside: I have no idea why I set the stage to needs patch.  The API discussion is needed first!)
msg212731 - (view) Author: Surya K (suryak) Date: 2014-03-04 17:32
Without really making big the in the API, one way of providing a position sensitive methods to enable more organized headers is to have. Having an index based api probably needs `more` discussion and thought!

1. add_header_before('existing-header-name', 'new-name', value)
2. add_header_after('existing-header-name', 'new-name', value)

The `existing-header-name` is again taken by first-found.

If put all routing, forwarded headers, one can always use `add_header_before('From'...)`.
msg273380 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-08-22 15:30
I've updated this to Python 3.6, but I don't know if there's time to design an API and implementation in the time left before beta 1.  But a use case has come up, so I want to reboot this discussion (yes, it should go to email-sig too).

Apparently ARC headers <http://arc-spec.org/> requires a header order.  In Mailman we have a GSoC student who is working on an implementation, and he's stuck because he needs to *prepend* ARC headers on the message, but the API gives him no direct way to do this.

We do have all the API problems that RDM points out.  I'm not going to suggest to the student that they use Message._headers.  It seems like the only way to do what he needs is to .get() all the headers in order, then del them all, manipulate the list out-of-band, and then re-add them all in the original order with the ARC headers at the front.  Yuck.
msg273381 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-08-22 15:32
I meant .items()
History
Date User Action Args
2022-04-11 14:57:46adminsetgithub: 62339
2016-08-22 15:32:58barrysetmessages: + msg273381
2016-08-22 15:30:25barrysetmessages: + msg273380
2016-08-22 15:20:53barrysetversions: + Python 3.6, - Python 3.4
2014-03-04 17:32:48suryaksetmessages: + msg212731
2014-03-03 15:46:10r.david.murraysetmessages: + msg212646
stage: needs patch ->
2014-03-03 15:42:50r.david.murraysetmessages: + msg212645
2014-03-03 15:03:50suryaksetfiles: + suryak.patch

nosy: + suryak
messages: + msg212642

keywords: + patch
2013-06-05 14:26:38r.david.murraysetmessages: + msg190662
2013-06-05 14:07:49barrysetmessages: + msg190661
2013-06-05 13:21:12r.david.murraysettype: behavior -> enhancement
title: email module's add_header appends instead of inserting -> email module should have a way to prepend and insert headers
components: + email

nosy: + barry, r.david.murray
versions: + Python 3.4, - Python 3.2
messages: + msg190658
stage: needs patch
2013-06-05 04:35:54kbandlacreate