Title: Pydoc: add option to remove run-specific ids (addresses)
Type: enhancement Stage: test needed
Components: Documentation Versions: Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: ammar2, asvetlov, docs@python, peteroupc, serhiy.storchaka, terry.reedy
Priority: normal Keywords:

Created on 2020-01-19 23:03 by peteroupc, last changed 2020-02-01 19:38 by terry.reedy.

Messages (9)
msg360278 - (view) Author: Peter O. (peteroupc) Date: 2020-01-19 23:03
It appears that if a method has default parameters set to functions, as in this example:

    def f1():

    def f2(a, b=f1):

The resulting Pydoc output produces a different, nondeterministic rendering for the f2 method each time it generates the documentation, such as `m1(a, b=<function f1 at 0x7f4ff67f8950>)` or `m1(a, b=<function f1 at 0x7f4ff67f8950>)`.  And this is problematic for version control systems, among other things, especially since this is not a meaningful change to the documentation.

One solution may be to write, say, `m1(a, b=f1)` instead.
msg360593 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-01-24 08:20
See also: which discusses changing the string representation of functions to omit the address.
msg360600 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-01-24 09:24
You will get the same problem for common idiom of using a singleton for optional parameters without default value.

    _singleton = object()
    def get(key, default=_signleton):
        if default is _signleton:

And for other objects whos repr contains "at 0x...".

And even more complex problem with sets.

When test the pydoc output, ignore non-deterministic parts. For example, doctest allows you to use a placeholder.

   >>> C() #doctest: +ELLIPSIS
   <__main__.C instance at 0x...>
msg360819 - (view) Author: Peter O. (peteroupc) Date: 2020-01-27 22:50
No, the use case I have in mind is storing outputs of the pydoc3 program -- as is -- in version control systems such as Git (e.g., running a command like "pydoc3 mymodule > mymodule_doc.txt").  The pydoc3 output is not further parsed by programs, or even "tested".

For example, adding "#doctest: +ELLIPSIS" as in the following example does not solve the problem in the opening post:

    def testfunc(a, m=d0): #doctest: +ELLIPSIS
msg360842 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-01-28 08:16
Serhiy is right, I doubt if we can do something here.
A function address is a part of its __repr__() output.
msg360843 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-01-28 08:29
Even if we change the repr of functions (I like this idea), this will not solve the general problem: many reprs contain an object id.

In your case I suggest to post-process the pydoc output and replace parts matching ' at 0x[0-9a-fA-F]+'.
msg361128 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-01-31 19:57
repr(f2) is not 'non-deterministic'.  If one runs it again for the same function object, one gets the same answer. It only changes when one starts over and creates a new function object with the same name but (usually) different id.

I think it appropriate that the default object representation include the object id.  It may help with debugging or other operations.

I agree that pydoc output with run-dependent ids may not be suitable for storage in version control.  Either remove or normalize them yourself or ask the pydoc developers to add an option to improve its output.

msg361151 - (view) Author: Peter O. (peteroupc) Date: 2020-02-01 05:56
Is this bug tracker the correct place to "ask the pydoc developers to add an option to improve [the Pydoc] output", in the sense that the option doesn't write out object IDs?  If not, where is the correct place to do so?
msg361188 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-02-01 19:38
Sorry, I was thinking of something else when I closed this.  What you want is a new -x option, for some 'x'.  The following would help, especially as there is no one who specifically maintains pydoc.

a) a specific proposal or set of proposals as to what letter to use for the option and what the result should be.

b) a currently failing test case (based on the above).

c. a patch for test.test_pydoc adding the test, preferably self-contained (no added file)  -- or does any current test already involve an 'at 0xnnnnnnnn' output?

d. a patch for making the test pass.
Date User Action Args
2020-02-01 19:38:02terry.reedysetstatus: closed -> open

title: Run-dependent Pydoc output for functions default parameters -> Pydoc: add option to remove run-specific ids (addresses)
messages: + msg361188
versions: + Python 3.9
type: behavior -> enhancement
resolution: third party ->
stage: resolved -> test needed
2020-02-01 05:56:44peteroupcsetmessages: + msg361151
2020-01-31 19:57:51terry.reedysetstatus: open -> closed

nosy: + terry.reedy
title: Nondeterministic Pydoc output on functions that have functions as default parameters -> Run-dependent Pydoc output for functions default parameters
messages: + msg361128

resolution: third party
2020-01-28 08:29:09serhiy.storchakasetmessages: + msg360843
2020-01-28 08:16:56asvetlovsetnosy: + asvetlov
messages: + msg360842
2020-01-27 22:50:43peteroupcsetstatus: closed -> open
resolution: not a bug -> (no value)
messages: + msg360819
2020-01-24 09:24:59serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg360600

resolution: not a bug
stage: resolved
2020-01-24 08:20:31ammar2setnosy: + ammar2
messages: + msg360593
2020-01-19 23:03:57peteroupccreate