classification
Title: help() fails for zlib Compress and Decompress objects
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: larry, ncoghlan, python-dev, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2014-02-02 17:12 by serhiy.storchaka, last changed 2014-02-05 13:11 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
issue20489_01.patch yselivanov, 2014-02-03 18:17 review
zlib_parameters_defaults.patch serhiy.storchaka, 2014-02-04 11:50 review
Messages (11)
msg210001 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-02 17:12
>>> import zlib
>>> d = zlib.decompressobj()
>>> d
<zlib.Decompress object at 0xb70a9288>
>>> c = zlib.compressobj()
>>> c
<zlib.Compress object at 0xb6f85ac8>
>>> zlib.DEF_BUF_SIZE
16384
>>> zlib.Z_FINISH
4
>>> help(c)
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2243, in wrap_value
    value = eval(s, module_dict)
  File "<string>", line 1, in <module>
NameError: name 'Z_FINISH' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2246, in wrap_value
    value = eval(s, sys_module_dict)
  File "<string>", line 1, in <module>
NameError: name 'Z_FINISH' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/_sitebuiltins.py", line 99, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1799, in __call__
    self.help(request)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1849, in help
    else: doc(request, 'Help on %s:', output=self._output)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1585, in doc
    pager(render_doc(thing, title, forceload))
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1578, in render_doc
    return title % desc + '\n\n' + renderer.document(object, name)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 357, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1276, in docclass
    lambda t: t[1] == 'method')
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1225, in spill
    name, mod, object))
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 358, in document
    if inspect.isroutine(object): return self.docroutine(*args)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1328, in docroutine
    signature = inspect.signature(object)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 1695, in signature
    return Signature.from_builtin(obj)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2298, in from_builtin
    p(name, default)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2284, in p
    default_node = RewriteSymbolics().visit(default_node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 245, in visit
    return visitor(node)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2276, in visit_Name
    return wrap_value(node.id)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2248, in wrap_value
    raise RuntimeError()
RuntimeError
>>> help(d)
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2243, in wrap_value
    value = eval(s, module_dict)
  File "<string>", line 1, in <module>
NameError: name 'DEF_BUF_SIZE' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2246, in wrap_value
    value = eval(s, sys_module_dict)
  File "<string>", line 1, in <module>
NameError: name 'DEF_BUF_SIZE' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/_sitebuiltins.py", line 99, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1799, in __call__
    self.help(request)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1849, in help
    else: doc(request, 'Help on %s:', output=self._output)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1585, in doc
    pager(render_doc(thing, title, forceload))
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1578, in render_doc
    return title % desc + '\n\n' + renderer.document(object, name)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 357, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1276, in docclass
    lambda t: t[1] == 'method')
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1225, in spill
    name, mod, object))
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 358, in document
    if inspect.isroutine(object): return self.docroutine(*args)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1328, in docroutine
    signature = inspect.signature(object)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 1695, in signature
    return Signature.from_builtin(obj)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2298, in from_builtin
    p(name, default)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2284, in p
    default_node = RewriteSymbolics().visit(default_node)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 245, in visit
    return visitor(node)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2276, in visit_Name
    return wrap_value(node.id)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 2248, in wrap_value
    raise RuntimeError()
RuntimeError
msg210097 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-02-03 08:34
The problem is that 'zlib.compressobj().flush.__module__ is None'
Can we fix that in zlib?
msg210136 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-03 15:10
Nope, we can't fix it in zlib.  As far as I can tell, the problem is that method_get() in descrobject.c passes in NULL for the module to PyCFunction_NewEx().  method_get gets the type as part of the descriptor protocol, but type objects in C don't appear to have a module pointer.  Module objects in Python do seem to, so maybe we can get it from somewhere?

If we can't fix this, we'll have to go back to expressly writing the name of the module in the expression ("zlib.Z_FINISH").  I assume changing the expression to use Z_FINISH (etc) makes it work again?

I'm calling in Nick, maybe he'll know what to do.

Nick: default values in text signatures allow simple expressions. inspect.Signature looks up the values of constant-ish things like "sys.maxsize".  It allows you to name constants from the local module without the module name first; it gets the module using the __module__ attribute on the callable.
msg210149 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-02-03 16:56
One way, albeit ugly, would be to further extend __text_signature__ format, to allow something like:

   sig=(...);module=zlib
msg210151 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-02-03 17:39
> One way, albeit ugly, would be to further extend 
> __text_signature__ format, to allow something like:
>
>   sig=(...);module=zlib

OR

   sig=zlib.Compress.flush(...)
msg210152 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-02-03 18:17
Attached is an experimental patch (issue20489_01.patch) that implements the idea of encoding full function name in its 'sig=' spec.

Test file:

   import zlib
   import inspect

   print(zlib.compressobj().flush.__text_signature__)
   print(inspect.signature(zlib.compressobj().flush))

Output:

   zlib.Compress.flush($self, mode=Z_FINISH)
   (mode=4)

And 'help(zlib.compressobj())' also works as expected.

If you like this approach I can fine-tune the patch.
msg210203 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-04 11:20
For 3.4, I'd prefer to just not convert these functions. The right fix is to figure out how to get __name__ set appropriately, and that's something the C extension module improvements in 3.5 should be able to help with.
msg210204 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-04 11:24
For example, see the builtins patch on issue 20184 where I initially converted sorted() to AC, but then found that making it *work* as an AC function was actually quite difficult due to the PyList implementation expecting to be given a arg tuple and kwds dict.

Something like that seems appropriate here as well - break the magic formatting of the clinic input, add a reference to this bug and then tweak the signature line manually to not be a valid AC signature line.
msg210208 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-04 11:50
Expressly writing the name of the module in the expression solves this issue.
msg210213 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-04 12:05
If qualifying the expression explicitly does the trick, that sounds good, too :)
msg210302 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-02-05 11:43
New changeset 31ddd45a05ba by Serhiy Storchaka in branch 'default':
Issue #20489: Explicitly qualified expressions for default values in methods.
http://hg.python.org/cpython/rev/31ddd45a05ba
History
Date User Action Args
2014-02-05 13:11:04serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-02-05 11:43:04python-devsetnosy: + python-dev
messages: + msg210302
2014-02-04 12:05:39ncoghlansetmessages: + msg210213
2014-02-04 11:50:41serhiy.storchakasetfiles: + zlib_parameters_defaults.patch

messages: + msg210208
stage: patch review
2014-02-04 11:24:37ncoghlansetmessages: + msg210204
2014-02-04 11:20:41ncoghlansetmessages: + msg210203
2014-02-03 18:17:35yselivanovsetfiles: + issue20489_01.patch
keywords: + patch
messages: + msg210152
2014-02-03 17:39:48yselivanovsetmessages: + msg210151
2014-02-03 16:56:16yselivanovsetmessages: + msg210149
2014-02-03 15:10:12larrysetnosy: + ncoghlan
messages: + msg210136
2014-02-03 08:34:14yselivanovsetmessages: + msg210097
2014-02-02 17:12:02serhiy.storchakacreate