Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug in python >= 2.7 with urllib2 fragment #55912

Closed
IvanIvanenko mannequin opened this issue Mar 28, 2011 · 11 comments
Closed

Bug in python >= 2.7 with urllib2 fragment #55912

IvanIvanenko mannequin opened this issue Mar 28, 2011 · 11 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@IvanIvanenko
Copy link
Mannequin

IvanIvanenko mannequin commented Mar 28, 2011

BPO 11703
Nosy @orsenthil, @asvetlov
Files
  • issue11703_py27.patch: Patch against 2.7
  • issue11703_py31.patch: Patch against 3.1
  • issue11703_py27_with_redirect.patch: Patch against 2.7
  • issue11703_py31_with_redirect.patch: Patch against 3.1
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/orsenthil'
    closed_at = <Date 2011-04-12.23:38:52.024>
    created_at = <Date 2011-03-28.21:23:30.472>
    labels = ['type-bug', 'library']
    title = 'Bug in python >= 2.7 with urllib2 fragment'
    updated_at = <Date 2011-04-13.01:47:53.941>
    user = 'https://bugs.python.org/IvanIvanenko'

    bugs.python.org fields:

    activity = <Date 2011-04-13.01:47:53.941>
    actor = 'python-dev'
    assignee = 'orsenthil'
    closed = True
    closed_date = <Date 2011-04-12.23:38:52.024>
    closer = 'orsenthil'
    components = ['Library (Lib)']
    creation = <Date 2011-03-28.21:23:30.472>
    creator = 'Ivan.Ivanenko'
    dependencies = []
    files = ['21456', '21457', '21555', '21556']
    hgrepos = []
    issue_num = 11703
    keywords = ['patch']
    message_count = 11.0
    messages = ['132423', '132497', '132508', '132613', '133183', '133620', '133621', '133624', '133625', '133634', '133635']
    nosy_count = 5.0
    nosy_names = ['orsenthil', 'asvetlov', 'santoso.wijaya', 'python-dev', 'Ivan.Ivanenko']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue11703'
    versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3']

    @IvanIvanenko
    Copy link
    Mannequin Author

    IvanIvanenko mannequin commented Mar 28, 2011

    result = urllib.urlopen("http://docs.python.org/library/urllib.html#OK")
    print result.geturl()
    
    result = urllib2.urlopen("http://docs.python.org/library/urllib.html#OK")
    print result.geturl()

    Python 2.6 returns:
    "http://docs.python.org/library/urllib.html#OK"
    "http://docs.python.org/library/urllib.html#OK"

    Python 2.7 returns:
    "http://docs.python.org/library/urllib.html#OK"
    "http://docs.python.org/library/urllib.html"

    2to3 -w test.py
    Python 3 returns:
    "http://docs.python.org/library/urllib.html"
    "http://docs.python.org/library/urllib.html"

    I expect geturl() result with "#OK" in all cases

    @IvanIvanenko IvanIvanenko mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 28, 2011
    @santosowijaya
    Copy link
    Mannequin

    santosowijaya mannequin commented Mar 29, 2011

    This is because the Request class' constructor splits the URL into __original and fragment:

        def __init__(self, url, data=None, headers={},
                     origin_req_host=None, unverifiable=False):
            # unwrap('<URL:type://host/path>') --> 'type://host/path'
            self.__original = unwrap(url)
            self.__original, fragment = splittag(self.__original)

    And the construction of object that urlopen() returns has its geturl() returns the request object's __original field (by now, minus the fragment).

    @santosowijaya
    Copy link
    Mannequin

    santosowijaya mannequin commented Mar 29, 2011

    Attaching patches against 2.7 and 3.1 branches.

    @IvanIvanenko
    Copy link
    Mannequin Author

    IvanIvanenko mannequin commented Mar 30, 2011

    Santa4nt, I think you also need to check case with Redirect Response URL:

    print urllib2.urlopen("http://16.foobnix-cms.appspot.com/test_base").geturl()

    python 2.6 returns OK
    http://16.foobnix-cms.appspot.com/test_redirect#json={value:'OK'}

    python 2.7 returns KO
    http://16.foobnix-cms.appspot.com/test_redirect

    @santosowijaya
    Copy link
    Mannequin

    santosowijaya mannequin commented Apr 6, 2011

    It already does. ;-)

    Python 2.7.1+ (default, Apr  6 2011, 16:25:38) [MSC v.1500 32 bit (Intel)] on wi
    n32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import urllib2
    [74578 refs]
    >>> fp = urllib2.urlopen('http://16.foobnix-cms.appspot.com/test_base')
    [75643 refs]
    >>> fp.geturl()
    "http://16.foobnix-cms.appspot.com/test_redirect#json={value:'OK'}"
    [75645 refs]

    I'm attaching patches with the appropriate unittest for the redirected case, though.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 12, 2011

    New changeset 3f240a1cd245 by Senthil Kumaran in branch '3.1':
    Fix bpo-11703 - urllib2.geturl() does not return correct url when the original url contains #fragment. Patch Contribution by Santoso Wijaya.
    http://hg.python.org/cpython/rev/3f240a1cd245

    @orsenthil
    Copy link
    Member

    It should be noted that the bug surfaced in 2.7 and above due to changes made as part of bpo-8280.

    @orsenthil orsenthil self-assigned this Apr 12, 2011
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 12, 2011

    New changeset 6e73f75ee034 by Senthil Kumaran in branch '2.7':
    Fix bpo-11703 - urllib2.get_url does not handle fragment in url properly.
    http://hg.python.org/cpython/rev/6e73f75ee034

    @orsenthil
    Copy link
    Member

    This is fixed in all the codelines. Thanks for the patch, Santoso.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 13, 2011

    New changeset 8ee48ec69844 by Senthil Kumaran in branch '3.1':
    Update the News for the fix to bpo-11703.
    http://hg.python.org/cpython/rev/8ee48ec69844

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 13, 2011

    New changeset 502bb809b03b by Senthil Kumaran in branch '2.7':
    update news in 2.7 for Issue bpo-11703
    http://hg.python.org/cpython/rev/502bb809b03b

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant