This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Explict proxies for urllib.urlopen()
Type: Stage:
Components: Library (Lib) Versions: Python 2.2
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: fdrake Nosy List: aimacintyre, fdrake, gimbo
Priority: normal Keywords: patch

Created on 2002-02-27 14:53 by gimbo, last changed 2022-04-10 16:05 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
urllib_proxies_patch.diff gimbo, 2002-02-27 14:53 Patch to urllib.py and documentation - allow explicit proxy specifcation in urllib.urlopen()
urllib_proxies_patch.cdiff gimbo, 2002-03-04 09:33
urllib_proxies_docs.cdiff gimbo, 2002-03-21 11:08 liburllib.tex patch implementing aimacintyre's suggestions
Messages (7)
msg39100 - (view) Author: Andy Gimblett (gimbo) Date: 2002-02-27 14:53
This patch extends urllib.urlopen() so that
proxies may be specified explicitly.  This is
achieved by adding an optional "proxies"
parameter.  If this parameter is omitted,
urlopen() acts exactly as before, ie gets
proxy settings from the environment.

This is useful if you want to tell urlopen()
not to use the proxy: just pass an empty
dictionary.

Also included is a patch to the urllib
documentation explaining the new parameter.

Apologies if patch format is not exactly as
required: this is my first submission.  All
feedback appreciated.  :-)
msg39101 - (view) Author: Andrew I MacIntyre (aimacintyre) * (Python triager) Date: 2002-03-03 03:32
Logged In: YES 
user_id=250749

Having just looked at this myself, I can understand where you're coming from, however my reading between the lines of the 
docs is that if you care about the proxies then you are supposed to use urllib.FancyURLopener (or urllib.URLopener) 
directly.  If this is the intent, the docs could be a little clearer about this.
msg39102 - (view) Author: Andrew I MacIntyre (aimacintyre) * (Python triager) Date: 2002-03-03 03:34
Logged In: YES 
user_id=250749

BTW, the patch guidelines indicate a strong preference for context diffs with unified diffs a poor second.
msg39103 - (view) Author: Andy Gimblett (gimbo) Date: 2002-03-04 09:33
Logged In: YES 
user_id=262849

Thanks for feedback re: diffs.  Have now found out
about context diffs and attached new version - hope
this is better.

Regarding the patch itself, this arose out of a newbie
question on c.l.py and I was reminded that this was an
issue I'd come across in my early days too.  Personally
I'd never picked up the hint that you should use
FancyURLopener directly.

If preferred, I could have a go at patching the docs
to make that clearer?
msg39104 - (view) Author: Andrew I MacIntyre (aimacintyre) * (Python triager) Date: 2002-03-10 05:31
Logged In: YES 
user_id=250749

I think expanding the docs is the go here.

In looking at the 2.2 docs (11.4 urllib), the bits that I think could usefully be improved include:-
- the paragraph describing the proxy environment variables should note that on Windows,
  browser (at least for InternetExplorer - I don't know about Netscape) registry settings for proxies
  will be used when available;
- a short para noting that proxies can be overridden using URLopener/FancyURLopener 
  class instances, documented further down the page, placed just before the note about 
  not supporting authenticating proxies;
- adding a description of the "proxies" parameter to the URLopener class definition;
- adding an example of bypassing proxies to the examples subsection (11.4.2).

If/when you upload a doc patch, I suggest that you assign it to Fred Drake, who is the 
chief docs person.
msg39105 - (view) Author: Andy Gimblett (gimbo) Date: 2002-03-21 11:08
Logged In: YES 
user_id=262849

OK, have updated docs as suggested by aimacintyre,
attached as urllib_proxies_docs.cdiff

I also added an example for explicit proxy
specification, since it illustrates how the proxies
dictionary should be structured.
msg39106 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2002-04-04 20:51
Logged In: YES 
user_id=3066

I've checked this in, with some changes to the code for
urlopen().  When a proxy configuration is supplied, the
version I checked in does not save the opener if there isn't
one; it always discards it.  If you really want to use a
specific proxy configuration with the simple functions,
create the opener and assign it to urllib._urlopener.
History
Date User Action Args
2022-04-10 16:05:02adminsetgithub: 36173
2002-02-27 14:53:42gimbocreate