classification
Title: urllib data URL
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: docs@python, orsenthil, panzi, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2012-11-07 03:28 by panzi, last changed 2012-11-24 18:39 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
doc-urllib.request-data-url-recipe.patch panzi, 2012-11-07 03:27 Patch that adds a data URL handler recipe to the urllib documentation. review
urllib.request-data-url.patch panzi, 2012-11-18 04:41 urllib.request patch for data url support (w/ doc and tests) review
Repositories containing patches
https://bitbucket.org/panzi/cpython
Messages (15)
msg175041 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-07 03:27
I think it would be really helpful if urllib would support data URLs. However, I was told on the python-ideas mailing list that it would probably only added as recipe in the documentation. The attached patch adds such an recipe to the urllib.request documentation. I've never written a doc patch (or any patch) for python before, so I don't know if I added it at the right place in the right format.
msg175667 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-16 03:45
New patch. Instead of adding the data URL support to the doc as a recipe I added it to urllib.request directly. I think this is better and justified, because the old legacy URLopener had (some kind) of support for data URLs.

OT: I think that the legacy URLopener has bugs. It only unquotes the data if it's base64 encoded, but it should be unquoted in any case. Also it returns a StringIO and decodes the content as latin-1. This is wrong, the content is a binary string. It could be text encoded in any kind of charset or not text at all (e.g. an image).
msg175828 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-17 23:36
A couple of comments:

- the patch needs a test (and docs too)
- are you sure ignoring POSTed data is the right thing to do? Shouldn't we forbid it instead?
- I think it would be nice to reference the RFC number somewhere
- not sure why you raise IOError on a bad URL; I would say ValueError is the right exception here

+1 on the general idea, by the way.
msg175831 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-17 23:42
On 11/18/2012 12:36 AM, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
> A couple of comments:
>
> - the patch needs a test (and docs too)

Will do (when I have time).

> - are you sure ignoring POSTed data is the right thing to do? Shouldn't we forbid it instead?
> - I think it would be nice to reference the RFC number somewhere
> - not sure why you raise IOError on a bad URL; I would say ValueError is the right exception here
>

I did that because that's what the old URLopener code does (ignoring POSTed data and raising an 
IOError). The comment is actually a 1:1 copy of the old code.

> +1 on the general idea, by the way.
>
> ----------
> nosy: +pitrou
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue16423>
> _______________________________________
>
msg175833 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-17 23:43
> > - are you sure ignoring POSTed data is the right thing to do? Shouldn't we forbid it instead?
> > - I think it would be nice to reference the RFC number somewhere
> > - not sure why you raise IOError on a bad URL; I would say ValueError is the right exception here
> >
> 
> I did that because that's what the old URLopener code does (ignoring POSTed data and raising an 
> IOError). The comment is actually a 1:1 copy of the old code.

Ok. I think it's better to take a fresh start here :-)
msg175840 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-18 01:31
On 11/18/2012 12:36 AM, Antoine Pitrou wrote:
>
> - the patch needs a test (and docs too)
> - are you sure ignoring POSTed data is the right thing to do? Shouldn't we forbid it instead?

Btw.: The file:// protocol handler also just ignores posted data, so I think the data url handler 
shouldn't deviate from that. Either both raise an error when data is posted or none.
msg175842 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-18 04:16
Ok, I've added a documentation and some tests. Is it ok this way? More tests? More/other documentation?
msg175867 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-18 11:38
> On 11/18/2012 12:36 AM, Antoine Pitrou wrote:
> >
> > - the patch needs a test (and docs too)
> > - are you sure ignoring POSTed data is the right thing to do? Shouldn't we forbid it instead?
> 
> Btw.: The file:// protocol handler also just ignores posted data, so I think the data url handler 
> shouldn't deviate from that. Either both raise an error when data is posted or none.

Ah, fair enough, then. Thanks for the updated patch, I'll take a look
(unless someone beats me to it).
msg176294 - (view) Author: Roundup Robot (python-dev) Date: 2012-11-24 17:00
New changeset a182367eac5a by Antoine Pitrou in branch 'default':
Issue #16423: urllib.request now has support for ``data:`` URLs.
http://hg.python.org/cpython/rev/a182367eac5a
msg176295 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-24 17:03
I've committed your patch after having made the few very minor changes mentioned in the review. Thank you very much!
msg176298 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-24 17:23
Great!

Feels awesome to have my first bit of code contributed to the Python project. :)
msg176299 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-24 17:49
You're welcome! We're always happy to have new contributors.

I've forgotten something: could you sign a contributor agreement?
You'll find instructions at http://www.python.org/psf/contrib/
msg176300 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-24 17:55
Will do (later today).
msg176304 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-24 18:30
Hmm, which of the two initial licenses should I choose? Which one do you rather want me to choose?
msg176305 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-24 18:39
> Hmm, which of the two initial licenses should I choose? Which one do
> you rather want me to choose?

Whichever you prefer. They should be equivalent in their terms
(non-copyleft free licenses).
History
Date User Action Args
2012-11-24 18:39:10pitrousetmessages: + msg176305
2012-11-24 18:30:05panzisetmessages: + msg176304
2012-11-24 17:55:01panzisetmessages: + msg176300
2012-11-24 17:49:19pitrousetmessages: + msg176299
2012-11-24 17:23:55panzisetmessages: + msg176298
2012-11-24 17:03:24pitrousetstatus: open -> closed
messages: + msg176295

assignee: orsenthil ->
resolution: not a bug
stage: patch review -> resolved
2012-11-24 17:00:13python-devsetnosy: + python-dev
messages: + msg176294
2012-11-18 11:38:27pitrousetmessages: + msg175867
2012-11-18 04:42:11panzisetfiles: - urllib.request-data-url.patch
2012-11-18 04:41:47panzisetfiles: + urllib.request-data-url.patch
2012-11-18 04:41:33panzisetfiles: - urllib.request-data-url.patch
2012-11-18 04:16:02panzisetfiles: + urllib.request-data-url.patch

messages: + msg175842
2012-11-18 01:31:15panzisetmessages: + msg175840
2012-11-17 23:43:19pitrousetmessages: + msg175833
2012-11-17 23:42:08panzisetmessages: + msg175831
2012-11-17 23:36:39pitrousetnosy: + pitrou
messages: + msg175828
2012-11-16 08:19:04serhiy.storchakasetstage: patch review
versions: - Python 3.1, Python 3.2, Python 3.3, Python 3.5
2012-11-16 03:50:36panzisettype: enhancement
components: + Library (Lib), - Documentation
title: urllib data URL recipe -> urllib data URL
2012-11-16 03:45:11panzisetfiles: + urllib.request-data-url.patch

messages: + msg175667
2012-11-07 05:08:37orsenthilsetassignee: docs@python -> orsenthil

nosy: + orsenthil
2012-11-07 03:28:02panzicreate