Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pydoc generates invalid docs.python.org link for xml.etree.ElementTree and other modules #60688

Closed
mgedmin mannequin opened this issue Nov 16, 2012 · 33 comments
Closed
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@mgedmin
Copy link
Mannequin

mgedmin mannequin commented Nov 16, 2012

BPO 16484
Nosy @brettcannon, @birkenfeld, @terryjreedy, @mgedmin, @merwok, @bitdancer, @ericsnowcurrently, @vadmium, @kau8hik
Files
  • issue16484_python2.7.patch: fixed patch for python 2.7
  • issue16484_python3.2.patch: patch for pydoc module docs link (python 3.2 version)
  • issue16484_python3.3.patch: patch for pydoc module docs link (python 3.3 version)
  • Issue16484_rev_python3.6.patch: patch for pydoc module docs link with tests(python 3.6 version)
  • issue16484_test_python3.6.patch: patch for pydoc module tests(python 3.6 version)
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-06-12.08:00:55.672>
    created_at = <Date 2012-11-16.10:46:28.913>
    labels = ['type-bug', 'tests']
    title = 'pydoc generates invalid docs.python.org link for xml.etree.ElementTree and other modules'
    updated_at = <Date 2016-06-12.08:00:55.671>
    user = 'https://github.com/mgedmin'

    bugs.python.org fields:

    activity = <Date 2016-06-12.08:00:55.671>
    actor = 'martin.panter'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-06-12.08:00:55.672>
    closer = 'martin.panter'
    components = ['Tests']
    creation = <Date 2012-11-16.10:46:28.913>
    creator = 'mgedmin'
    dependencies = []
    files = ['34236', '34237', '34238', '43160', '43355']
    hgrepos = []
    issue_num = 16484
    keywords = ['patch']
    message_count = 33.0
    messages = ['175675', '175706', '175713', '175784', '177029', '177035', '212155', '212156', '212220', '212297', '212298', '212309', '212317', '212321', '213601', '214819', '216663', '216706', '216709', '266970', '267137', '267174', '267186', '267188', '267242', '268090', '268166', '268188', '268315', '268321', '268323', '268331', '268333']
    nosy_count = 11.0
    nosy_names = ['brett.cannon', 'georg.brandl', 'terry.reedy', 'mgedmin', 'eric.araujo', 'r.david.murray', 'python-dev', 'eric.snow', 'martin.panter', 'sean.rodman', 'kau8hik']
    pr_nums = []
    priority = 'high'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue16484'
    versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

    @mgedmin
    Copy link
    Mannequin Author

    mgedmin mannequin commented Nov 16, 2012

    Do this:

    pydoc2.7 xml.etree.ElementTree

    then click on the MODULE DOCS link, which is http://docs.python.org/library/xml.etree.ElementTree

    You're redirected to http://docs.python.org/2/library/xml.etree.ElementTree which is a 404 page.

    @mgedmin mgedmin mannequin assigned docspython Nov 16, 2012
    @mgedmin mgedmin mannequin added the docs Documentation in the Doc dir label Nov 16, 2012
    @birkenfeld
    Copy link
    Member

    The redirect is not the problem. The page never existed.

    @mgedmin
    Copy link
    Mannequin Author

    mgedmin mannequin commented Nov 16, 2012

    A working link is http://docs.python.org/2/library/xml.etree.elementtree.html or http://docs.python.org/2/library/xml.etree.elementtree (with no .html at the end).

    Looks like capitalization is causing a problem.

    Did these pydoc links ever work?

    Would it be possible to make them work?

    There's a limited number of standard library modules with names that contain capital letters. find /usr/lib/python2.7 -name dist-packages -prune -o -name '*[A-Z]*' | wc -l gives me 211 names, but this is an imprecise upper bound because it includes files like 'distutils/README' that aren't modules or packages.

    Can I do anything to help?

    @merwok
    Copy link
    Member

    merwok commented Nov 17, 2012

    A patch and test making sure that pydoc generates the right URIs for upper-case modules would help.

    @merwok merwok added the type-bug An unexpected behavior, bug, or error label Nov 17, 2012
    @merwok merwok changed the title Missing/broken documentation redirect for http://docs.python.org/library/xml.etree.ElementTree pydoc generates invalid docs.python.org link for xml.etree.ElementTree and other modules Nov 17, 2012
    @mgedmin
    Copy link
    Mannequin Author

    mgedmin mannequin commented Dec 6, 2012

    Considering many existing Python installations out there would it be possible to fix this on the server side? I.e. lowercase the URL while redirecting?

    @birkenfeld
    Copy link
    Member

    Adding a redirect should be easy, yes.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 25, 2014

    I could try to create the patch for pydoc if you would like for me to.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 25, 2014

    To display my ignorance, I have run the pydoc command listed in the original message but I can't actually see where it lists the url. Or even where it has a link.

    @mgedmin
    Copy link
    Mannequin Author

    mgedmin mannequin commented Feb 25, 2014

    Near the top:

    Help on module xml.etree.ElementTree in xml.etree:

    NAME
    xml.etree.ElementTree

    FILE
    /usr/lib/python2.7/xml/etree/ElementTree.py

    MODULE DOCS
    http://docs.python.org/library/xml.etree.ElementTree

    DESCRIPTION
    ...

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 26, 2014

    Here is a working patch for python 2.7. all it does is lowercase the module name, but once I did that and clicked the link it worked correctly.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 26, 2014

    Note: It doesn't change the actual module name. Just how it is represented in the link.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 26, 2014

    Here is the patch for python 3.2. It implements the same fix that the 2.7 patch does.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 26, 2014

    Sorry guys, I missed a place I needed to add the lower() fuction to the module.__name__. Here is a fixed patch for python2.7.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Feb 26, 2014

    Here are the python3.2 and python3.3 patches. Please let me know if there is anything I need to change on these.

    @seanrodman
    Copy link
    Mannequin

    seanrodman mannequin commented Mar 14, 2014

    Is there anything else I can do for this?

    @merwok
    Copy link
    Member

    merwok commented Mar 25, 2014

    It would be nice to have unit tests for this change.

    Did you get an email from the review system? If not, follow the “review” link on the right of the list of files on this page.

    @vadmium
    Copy link
    Member

    vadmium commented Apr 17, 2014

    I don’t think these ones could be so easily fixed, but on my computer “pydoc” references:

    • library/importlib.machinery.html (ideally should be library/importlib.html#module-importlib.machinery)
    • library/tkinter.font.html (not in Python documentation at all that I am aware, except for brief mention in §25.1.1)

    @merwok
    Copy link
    Member

    merwok commented Apr 17, 2014

    Brett, any opposition to moving the doc about importlib submodules to separate files?

    @brettcannon
    Copy link
    Member

    No, I have no objections.

    @kau8hik
    Copy link
    Mannequin

    kau8hik mannequin commented Jun 2, 2016

    Applied patch from sean.rodman.
    Created a test.

    @kau8hik
    Copy link
    Mannequin

    kau8hik mannequin commented Jun 3, 2016

    Applied patch from sean.rodman.
    Created a tests.
    Incorporated comments from rdmurray

    @bitdancer
    Copy link
    Member

    This patch looks good to me. Note that the reason for the basedir change is that the test would fail if run from a build directory which hadn't been installed. An implication of this is that 'doc_loc' is None most of the time when the other tests are run, meaning that this new test is the only one that consistently tests the body of getdocloc. I think that method could use some more tests, but that is a separate issue.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jun 3, 2016

    New changeset 64e7c8531131 by R David Murray in branch '3.5':
    bpo-16484: Fix pydoc doc links to modules whose names are mixed case.
    https://hg.python.org/cpython/rev/64e7c8531131

    New changeset 912b9aad0b45 by R David Murray in branch 'default':
    Merge: bpo-16484: Fix pydoc doc links to modules whose names are mixed case.
    https://hg.python.org/cpython/rev/912b9aad0b45

    @bitdancer
    Copy link
    Member

    Thanks Sean and Kaushik.

    The final patch didn't apply to 2.7. If someone wants to port the patch I'll apply it, but I'm going to close the issue.

    @bitdancer bitdancer removed the type-bug An unexpected behavior, bug, or error label Jun 3, 2016
    @vadmium
    Copy link
    Member

    vadmium commented Jun 4, 2016

    It seems the test is broken on Windows:

    http://buildbot.python.org/all/builders/x86%20Windows7%203.5/builds/915/steps/test/logs/stdio
    ======================================================================
    ERROR: test_mixed_case_module_names_are_lower_cased (test.test_pydoc.PydocDocTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "D:\cygwin\home\db3l\buildarea\3.5.bolen-windows7\build\lib\test\test_pydoc.py", line 458, in test_mixed_case_module_names_are_lower_cased
        self.assertIn('xml.etree.elementtree', doc_link)
      File "D:\cygwin\home\db3l\buildarea\3.5.bolen-windows7\build\lib\unittest\case.py", line 1076, in assertIn
        if member not in container:
    TypeError: argument of type 'NoneType' is not iterable

    @vadmium vadmium reopened this Jun 4, 2016
    @vadmium
    Copy link
    Member

    vadmium commented Jun 10, 2016

    Kaushik or David (or anyone else), does you have an idea of how to fix the Windows tests? It would be good to make the buildbots healthy again, then it is easier to see new failures in Windows.

    @terryjreedy
    Copy link
    Member

    Those are the only failures on 2 of the 4 buildbots running gui tests.

    @vadmium
    Copy link
    Member

    vadmium commented Jun 11, 2016

    My guess is the file.startswith(basedir) in the getdocloc() method is getting tricked by non-canonical paths (e.g. lowercase vs uppercase C: drive?). Would it help to use something like os.path.normcase(), or maybe os.path.commonpath(), pathlib.PurePath.parts or PurePath.relative_to()?

    file = inspect.getabsfile(object)
    ...
    basedir = os.path.normcase(basedir)
    if (...
         (file.startswith(basedir) and ...

    Also, in the new get_pydoc_link() function, is that os.path.join() call meant to be there?

    basedir = os.path.join(dirname(dirname(__file__)))

    @kau8hik
    Copy link
    Mannequin

    kau8hik mannequin commented Jun 12, 2016

    • fixed broken test - test_mixed_case_module_names_are_lower_cased
    • tested on Windows, Mac, Ubuntu
    • thank you Martin Panter for suggestions.

    @kau8hik kau8hik mannequin added tests Tests in the Lib/test dir build The build process and cross-build and removed docs Documentation in the Doc dir labels Jun 12, 2016
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jun 12, 2016

    New changeset d84268ed62ac by Martin Panter in branch '3.5':
    Issue bpo-16484: Fix pydoc link and test on Windows, by Kaushik Nadikuditi
    https://hg.python.org/cpython/rev/d84268ed62ac

    New changeset 84d1e5a4b1c5 by Martin Panter in branch 'default':
    Issue bpo-16484: Merge pydoc Windows fixes from 3.5
    https://hg.python.org/cpython/rev/84d1e5a4b1c5

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jun 12, 2016

    New changeset c60d2e4f4f96 by Martin Panter in branch '2.7':
    Issue bpo-16484: PYTHONDOCS still uses “http:” in Python 2
    https://hg.python.org/cpython/rev/c60d2e4f4f96

    @berkerpeksag berkerpeksag added type-bug An unexpected behavior, bug, or error and removed build The build process and cross-build labels Jun 12, 2016
    @vadmium
    Copy link
    Member

    vadmium commented Jun 12, 2016

    Thanks Kaushik. I also changed the https:// detection to also detect http://, to maintain backwards compatibility.

    I am porting these changes to 2.7.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jun 12, 2016

    New changeset ede10138327e by Martin Panter in branch '2.7':
    bpo-16484: Change PYTHONDOCS to "https:", and fix links to use lowercase
    https://hg.python.org/cpython/rev/ede10138327e

    @vadmium vadmium closed this as completed Jun 12, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    7 participants