classification
Title: urllib: no_proxy variable values with leading dot not properly handled
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: xiang.zhang Nosy List: berker.peksag, martin.panter, python-dev, tloetzer, xiang.zhang, zvyn
Priority: normal Keywords: patch

Created on 2017-01-03 14:09 by tloetzer, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
ignore_dots-v1.patch zvyn, 2017-01-05 08:12 review
ignore_dots-v2.patch zvyn, 2017-01-05 23:07 removed unitended refactoring review
ignore_dots-v3.patch zvyn, 2017-01-08 08:15 review
Pull Requests
URL Status Linked Edit
PR 53 zvyn, 2017-01-05 03:58
PR 552 closed dstufft, 2017-03-31 16:36
Messages (10)
msg284565 - (view) Author: Thomas Loetzer (tloetzer) Date: 2017-01-03 14:09
Hi,

the fix for issue 26864 changed the behavior of urllib for no_proxy values with a leading dot to no longer match anything. This seems to be caused by always adding an additional dot between the hostname being checked and the entry.

Example:
no_proxy = '.company.internal'

I would expect 'somehost.company.internal' to be accessed without proxy, but this is not actually the case. Changing no_proxy to company.internal changes this and the host is accessed without proxy.

Python 2.7.11 and curl both handle this case fine and the proxy is not used.

Regards,
Thomas
msg284702 - (view) Author: Milan Oberkirch (zvyn) * Date: 2017-01-05 03:58
I added a line stripping dots from name, I think it makes sense to strip at the end as well (`example.com` is the same domain as `example.com.`).
msg284705 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-01-05 04:21
Note: We don't officially use pull requests yet. Current way of contributing to Python is documented at https://docs.python.org/devguide/patch.html
msg284714 - (view) Author: Milan Oberkirch (zvyn) * Date: 2017-01-05 08:12
Too bad, I was all exited when I saw that I can paste a PR link here :)

I downloaded the patch from the commit using github, I hope that works?
msg284717 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-01-05 09:36
Yes, thanks!
msg284811 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-06 11:05
> I think it makes sense to strip at the end as well (`example.com` is the same domain as `example.com.`).

Are your sure the host checked against the list is FQDN? With and without the trailing dot are different.
msg284967 - (view) Author: Milan Oberkirch (zvyn) * Date: 2017-01-08 08:15
Thanks for the feedback!

I changed the patch to use lstrip so that the FQDN "example.com." would match the NOPROXY entry "example.com.". I don't think we get FQDNs, but that's just my guess.

I think ideally "example.com" would also match "example.com." but that's a different issue.
msg285016 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-09 02:52
LGTM.
msg285019 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-09 03:57
New changeset cb4f73be9486 by Xiang Zhang in branch '2.7':
Issue #29142: Fix suffixes in no_proxy handling in urllib.
https://hg.python.org/cpython/rev/cb4f73be9486

New changeset 350c5a1f9cfd by Xiang Zhang in branch '3.5':
Issue #29142: Fix suffixes in no_proxy handling in urllib.
https://hg.python.org/cpython/rev/350c5a1f9cfd

New changeset e90efd9f203e by Xiang Zhang in branch '3.6':
Issue #29142: Merge 3.5.
https://hg.python.org/cpython/rev/e90efd9f203e

New changeset 3954f8f6be3d by Xiang Zhang in branch 'default':
Issue #29142: Merge 3.6.
https://hg.python.org/cpython/rev/3954f8f6be3d
msg285023 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-09 05:37
Thanks Milan and Thomas!
History
Date User Action Args
2017-03-31 16:36:22dstufftsetpull_requests: + pull_request961
2017-01-09 05:37:34xiang.zhangsetstatus: open -> closed
resolution: fixed
messages: + msg285023

stage: commit review -> resolved
2017-01-09 03:57:42python-devsetnosy: + python-dev
messages: + msg285019
2017-01-09 02:52:58xiang.zhangsetassignee: xiang.zhang
messages: + msg285016
stage: patch review -> commit review
2017-01-08 08:15:04zvynsetfiles: + ignore_dots-v3.patch

messages: + msg284967
2017-01-06 11:05:30xiang.zhangsetmessages: + msg284811
2017-01-06 00:17:12berker.peksagsettype: behavior
versions: + Python 3.5, Python 3.6, Python 3.7
2017-01-05 23:07:47zvynsetfiles: + ignore_dots-v2.patch
type: behavior -> (no value)
versions: - Python 3.5, Python 3.6, Python 3.7
2017-01-05 09:36:28berker.peksagsetversions: + Python 3.5, Python 3.6, Python 3.7
nosy: + martin.panter, xiang.zhang

messages: + msg284717

type: behavior
stage: patch review
2017-01-05 08:12:15zvynsetfiles: + ignore_dots-v1.patch
keywords: + patch
messages: + msg284714
2017-01-05 04:21:15berker.peksagsetnosy: + berker.peksag
messages: + msg284705
2017-01-05 03:58:33zvynsetnosy: + zvyn

messages: + msg284702
pull_requests: + pull_request8
2017-01-03 14:09:49tloetzercreate