classification
Title: typing.Tuple is class but is defined as data inside https://docs.python.org/3.6/objects.inv
Type: enhancement Stage: resolved
Components: Documentation, Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Bernát Gábor, docs@python, gvanrossum, levkivskyi
Priority: normal Keywords:

Created on 2017-07-25 11:17 by Bernát Gábor, last changed 2017-08-23 07:40 by levkivskyi. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3181 closed python-dev, 2017-08-22 14:23
Messages (15)
msg299048 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-07-25 11:17
Hello, 

typing.Tuple is class but is defined as data inside https://docs.python.org/3.6/objects.inv which means that Sphinx fails to find these.

Thanks,
msg299412 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-07-28 17:24
The fact that it is a class is an implementation detail and may change before Python 3.7 beta (situation is the same for Callable). Guido explicitly doesn't like to "advertise" it as a class yet. Unless he changed his mind, I would propose to close the issue, and instead consider treating this special case in Sphinx.
msg299457 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-07-29 06:31
I agree we should not advertise the type. However, whatever its type is should be in sync in what we put inside objects.inv. That's sort of a encoded binary black box for the user; the Sphinx tool included. It maps standard library elements to URLs. This ticket is about fix objects.inv to have the Tuple in the correct bucket.
msg299468 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-07-29 13:51
> This ticket is about fix objects.inv to have the Tuple in the correct bucket.

If you know how to do this without breaking the python docs system (so that Tuple and Callable will not have a "class" prefix on https://docs.python.org/3/library/typing.html) then I think it is OK.
msg299469 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-07-29 13:59
I would need to know who and how maintains the https://docs.python.org/3.6/objects.inv file. Do you have any idea?
msg299470 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-07-29 14:01
> Do you have any idea?

Unfortunately no.
msg300247 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-08-14 14:25
how can we find it out?
msg300263 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-08-14 19:22
Bernát, I would recommend asking this on Sphinx tracker (I also assigned this to docs@python since this seems to be a purely documentation issue).

https://github.com/sphinx-doc/sphinx
msg300713 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-08-22 17:21
Before I spend more time reviewing your patch, can you please explain what you mean by "sphinx fails to find these"? Is there a particular dead link on docs.python.org or a specific query you typed in the search box that failed to find the definition of typing.Tuple?
msg300721 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-08-22 20:55
so here's the problem in detail:

Intersphinx (http://www.sphinx-doc.org/en/stable/ext/intersphinx.html) is a built in Sphinx plugin that allows to link to documentation of other projects in your own documentation. For example when specifying the argument of a function which can be either int or string you might specify the argument to be Union[int, str]. In this case Sphinx will generate the type of the argument as this object, where e.g. the Union word will be a link to https://docs.python.org/3/library/typing.html#typing.Union and so on. 

In order to know for which object/function/type what is the correct URL an intersphinx object (which is basically a mapping) needs to be generated. This maps each element to an URL. This intersphinx mapping storage object (available at https://docs.python.org/3/objects.inv and similarly under other version pattern for the according version) is generated from the Python documentation. 

Now there's one indirection as far as this objects.inv goes; it does not actually stores mappings of element to URL, but what instead does it stores a mapping of type to element to url. Where type is here defined as either data/class etc. In order to resolve from an element to an URL both keys need to be correct (the type and the key too). 

For Python 3.6+ the typing.Tuple mapping now fails because according to the documentation the Tuple is under the section data, but once intersphinx looks at the Tuple during documentation generation it deduces it actually is of type class, and tries to locate it there. In order for intersphinx to resolve the correct web page URL, the runtime information of an element need to coincide with it's documentation type. Hence, what this PR tries to fix, migrating the documentation type/section of data to class (as in the meantime under Python 3.6 the Tuple is now a class).

For the end user the fact that this is a class is still hidden, but for the documentation generation tool to resolve the correct URL, the runtime information needs to coincide with the documentation one. We should probably add a unit test that makes sure all runtime "type" matches with documentation "type" in the future. Let's make that the scope of another PR; is now clear?
msg300724 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-08-22 22:24
> For the end user the fact that this is a class is still hidden

I am not sure what you mean by this, but with your PR the rendered docs will literally say ``class typing.Tuple``.

> We should probably add a unit test that makes sure all runtime "type" matches with documentation "type" in the future

I already mention, this was not an omission but a deliberate decision, see http://bugs.python.org/review/28644/diff/19105/Doc/library/typing.rst#newcode444 (and below the same for Callable)
msg300726 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-08-22 22:27
Thanks, I am a little closer to understanding now.

> once intersphinx looks at the Tuple during documentation generation it deduces it actually is of type class

How does intersphinx deduce this? Also, does it run when we generate the Python docs, or when you generate the docs for your project (that references the Python docs)? Finally, are there other objects in the typing for which this is a problem? (I guess I could answer this myself once I understand how intersphinx decides whether something is a class or data.)

I also think this is somewhat unfortunate, because it seems that whenever something is changed from "class" to "data" or vice versa, intersphinx will be confused, even though it's easy to give examples where such a change could be backwards compatible.
msg300741 - (view) Author: Bernát Gábor (Bernát Gábor) Date: 2017-08-23 05:07
I suppose when the python.org documentation is generated the objects.inv file gets generated with it (I did not found exactly this piece of code here though). When Sphinx runs, it's intersphinx plugin goes out to python.org, downloads the objects.inv, decodes it, and then tries to map the docstring param/return values/references to URLs. 

Actually intersphinx does not make the deduction of the type. It uses what the users entered in the docstring. In this case the user needs to know for each element to which bucket has been assigned to (e.g. class/data/function/method/exception/macro); and for compatibility reason that needs to stay stable, otherwise with a Python upgrade the user would need to update the code of its docstrings.

That being said for the sake of automation, in my case there actually another sphinx plugin (https://github.com/agronholm/sphinx-autodoc-typehints/blob/master/sphinx_autodoc_typehints.py) which actually generates, on the fly, the type information. So I suppose as a fix for my problem the sphinx_autodoc_typehint could be altered to still give back data, even though this now is a class. I'm not sure though how confident I am on "lying" to users about the data/class; but I suppose it's a necessary evil at this point.

So should we keep everything as it is? When I first identified why Tuple does not have the URL mapped to it, I thought the problem to be the fact that it goes to the wrong bucket (by just inspecting its type): data; but now I see that may not be such a bad thing after all.
msg300743 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-08-23 05:27
OK, I think Sphinx is way too complicated for its own good, and it's arguably not the fault of Python's documentation that this doesn't work for you.

I would like to close this issue as "won't fix", except... in a sense Tuple actually *is* a class, since you can subclass from e.g. Tuple[int, str].

@Levikvskyi, what do you think?
msg300744 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-08-23 07:40
TBH, I think this is a Sphinx problem, not a Python problem. And concerning ``Tuple`` being an actual class I think this is an implementation detail, so that I am closing this as "won't fix".
History
Date User Action Args
2017-08-23 07:40:00levkivskyisetstatus: open -> closed
resolution: wont fix
messages: + msg300744

stage: resolved
2017-08-23 05:27:48gvanrossumsetmessages: + msg300743
2017-08-23 05:07:38Bernát Gáborsetmessages: + msg300741
2017-08-22 22:27:47gvanrossumsetmessages: + msg300726
2017-08-22 22:24:55levkivskyisetmessages: + msg300724
2017-08-22 20:55:28Bernát Gáborsetmessages: + msg300721
2017-08-22 17:21:08gvanrossumsetmessages: + msg300713
2017-08-22 14:23:34python-devsetpull_requests: + pull_request3219
2017-08-14 19:22:48levkivskyisetnosy: + docs@python
messages: + msg300263

assignee: docs@python
components: + Documentation
type: enhancement
2017-08-14 14:25:58Bernát Gáborsetmessages: + msg300247
2017-07-29 14:01:45levkivskyisetmessages: + msg299470
2017-07-29 13:59:07Bernát Gáborsetmessages: + msg299469
2017-07-29 13:51:19levkivskyisetmessages: + msg299468
2017-07-29 06:31:56Bernát Gáborsetmessages: + msg299457
2017-07-28 17:24:34levkivskyisetnosy: + gvanrossum, levkivskyi
messages: + msg299412
2017-07-25 11:17:40Bernát Gáborcreate