classification
Title: Document lack of support for keyword arguments in C functions
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: open Resolution: accepted
Dependencies: Superseder:
Assigned To: docs@python Nosy List: BreamoreBoy, docs@python, eric.araujo, eric.smith, ezio.melotti, georg.brandl, l0nwlf, loewis, lukasz.langa, pitrou, rhettinger, schof, terry.reedy, twhitema
Priority: normal Keywords: patch

Created on 2010-04-08 22:02 by twhitema, last changed 2014-09-29 22:57 by BreamoreBoy.

Files
File name Uploaded Description Edit
posixmodule.patch l0nwlf, 2010-04-09 06:21 Tries to fix the doc command for posix.mkdir review
Messages (22)
msg102648 - (view) Author: Todd Whiteman (twhitema) Date: 2010-04-08 22:02
The doc command for os.mkdir is incorrect (at least for posix). It specifies that there is an optional mode keyword, but it's not a keyword argument, see below:

>>> import os
>>> help(os.mkdir)
mkdir(...)
    mkdir(path [, mode=0777])
    
    Create a directory.

>>> os.mkdir("/tmp/1", mode=777) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: mkdir() takes no keyword arguments


Suggest the following doc comment change:

mkdir(...)
    mkdir(path [, mode])
    
    Create a directory. Mode defaults to 0777 if not specified.
msg102683 - (view) Author: Shashwat Anand (l0nwlf) Date: 2010-04-09 06:21
Tested on trunk. 

11:46:02 l0nwlf-MBP:python-svn $ ./python.exe 
Python 2.7a4+ (trunk:79888M, Apr  9 2010, 11:41:22) 
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> posix.mkdir('1')
>>> posix.mkdir('2', 000)
>>> posix.mkdir('3', mode=000)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: mkdir() takes no keyword arguments
>>> import os
>>> os.mkdir('4', 777)
>>> os.mkdir('5', mode=777)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: mkdir() takes no keyword arguments
>>> 

The two options to fix this are:
1. Patch python-svn/Modules/posixmodule.c to take the keyword arguments.
2. Modify the PyDoc in posixmodule.c as Todd suggested.

I had attached a patch which solves the issue using solution '2'. 

Also I observed that os.makedirs() have no such issue even though it uses os.mkdir() because it uses 'mkdir(name, mode)' to call mkdir.
msg102713 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-04-09 12:34
The docstring looks right to me. "mode=0777" doesn't mean it takes a keyword argument, only that this argument has a default value.

You might want to fix posixmodule to accept keyword arguments, but it should probably be done for all functions then. An mkdir-specific patch would have little point IMO (after all it takes only two args, there's hardly any risk of confusion).
msg102714 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-04-09 12:35
Hello

This is a recurrent problem with Python functions implemented in C. Since introspection is not possible, developers have to use the first line of the docstring to write the signature, including default arguments. With your patch, people can’t rely on IDE tooltips that display introspected signature + first line of docstring anymore.

The ideal right fix would be for C functions to support introspection, e.g. with PEP 362.

Regards
msg102716 - (view) Author: Shashwat Anand (l0nwlf) Date: 2010-04-09 12:40
The ideal right fix would be for C functions to support introspection - Agreed, but then it will be needed to do so in quite a number of C codes.
msg102717 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-04-09 12:47
A more attainable fix would be a way to mark up positional-only arguments that have a default value.
msg104113 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-04-24 18:12
From http://docs.python.org/dev/reference/expressions.html#calls

“An implementation may provide built-in functions whose positional parameters do not have names, even if they are ‘named’ for the purpose of documentation, and which therefore cannot be supplied by keyword.”
msg104114 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-04-24 18:16
I think it would be overkill to add special markup for positional-only arguments. I think we should just close the issue as invalid.
msg104116 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-04-24 18:23
The comment I made before yours supports that. It’s not a bug, it’s a documented behavior that just needs more exposure in the intro to the docs. (Do we have a page explaining markup?)

The real real fix would be for C functions to accept kwargs, but as far as doc is concerned, it’s ok.
msg105366 - (view) Author: John Mark Schofield (schof) Date: 2010-05-09 03:08
Please don't close this as "invalid."

Most (all?) of the functions in the os module have positional-only arguments, which are documented in exactly the same manner as arguments which can be supplied using a keyword.

As someone reading the documentation, how am I supposed to know which arguments can be supplied with keywords and which can't?

This ticket and the link to http://docs.python.org/dev/reference/expressions.html#calls are helpful in explaining this now, but I would NEVER have thought to look there to find out why os.fdopen isn't working the way it's documented. Requiring me to experiment to determine which function works which way seems to miss the point of having documentation in the first place.

I take no position on whether this should be fixed with a documentation change or a code change, but it should be fixed. If the consensus is that a documentation change would be quicker to implement, I'm happy to submit a doc patch.
msg105367 - (view) Author: John Mark Schofield (schof) Date: 2010-05-09 03:15
I'd also suggest changing the title to "Documentation for many functions in os module is incomplete." I didn't because I don't know if that would be considered rude. (I'm new to the Python community.)
msg112071 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2010-07-30 12:41
Closing as invalid would not be wise because using named arguments as keywords is in Python taken for granted. Cases that are exceptions from this rule should be explicitly noted as to avoid confusion, especially for less experienced programmers.

Similar case for builtins: #9426
msg112078 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2010-07-30 13:39
Currently it's somewhat surprising that while the documentation often states the default values for certain builtins, invoking the builtin with keyword arguments as described in the documentation does not work.

Original discussion: #7447

I'm going through all builtins to see how the keyword arg support looks and I'm going to present a patch for Doc/library/builtins.rst that explicitly states that builtins don't support this kind of invocation.

Expect a patch for that case later today. As for `os` et al, the discussion goes on.
msg112079 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-30 13:43
You could copy this notice from reference/expressions#calls:

“An implementation may provide built-in functions whose positional parameters do not have names, even if they are ‘named’ for the purpose of documentation, and which therefore cannot be supplied by keyword.”

An addition to documenting/rest or documenting/markup would be useful too.
msg125017 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-01 23:59
Hmm, it may indeed be the best option to add a new directive option to say "this function does not take keyword args".  It would result in some form of unobtrusive but noticeable output in HTML.

It is a bit of an effort to add it everywhere it's necessary, but the most important instances (e.g. string methods) can be covered quickly, and it's fairly easy to grep the other instances (namely, grepping for METH_O and METH_VARARGS without METH_KEYWORDS).
msg125018 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-02 00:01
> Hmm, it may indeed be the best option to add a new directive option to
> say "this function does not take keyword args".  It would result in
> some form of unobtrusive but noticeable output in HTML.

Isn't it kind of a CPython-specific detail, though?
msg125019 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-02 00:03
Yes.  It's still an important detail; the explanation could say, "In CPython, this function does not take keyword args" and furthermore it's not really clear to me how much of the library reference applies to all Python implementations anyway.
msg125021 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-02 00:08
This is an implementation detail specific to CPython and subject to change.  I'm -1 on documenting it for every function/method and thereby making it part of the language spec. We've lived without this spec for almost twenty years, so I'm inclined to think it is truly unimportant.

It is sufficient to mention just once in the docs that CPython functions/methods sometimes don't take keywords.
msg125024 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-01-02 01:52
I think that the warning that things are not always as they seem should be repeated in the front of the library manual where the pseudo-arg names are actual used, so the library manual stands on its own. In any case, I believe a lot of people use the lib ref without reading and remembering every detail of the language ref.
msg125026 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-01-02 02:22
If there is no indication in the lib manual as to which parameter names and defaults are real and which are fake, then the safe guideline is to never use keywards for library functions and methods and always pass everything positionally.

Slightly more complicated is to be aware of which classes and modules are Python versus C coded. (If needed, change for module.py in /Lib.) For some modules, one can take a cue from doc examples that use keywords.

Otherwise, each person has to experiment for himself and  check each TypeError messages to determine whether it arises from a misspelling or a hidden limitation. And maybe go through the same process a year later after forgetting.

>"We've lived without this spec for almost twenty years,"
Yes, and people have been stumbling on this and complaining for probably just as long. Since []s are no longer used in the doc to indicate 'optional', they can and are being used to indicate 'position-only'. Specify in the introduction, where notation should be explained, that the limitation is only for current CPython and may be changed in the future or be different for other implementations. However....

In my opinion, the real solution is to remove the limitation. Since the language spec says args can be passed by keyword as well as by position, make it be that way for everything we distribute.
msg125034 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-02 08:25
> Isn't it kind of a CPython-specific detail, though?

If other implementations do provide proper keyword arguments,
I'd be skeptical that they all settled on the names that the
library documentation gives to the arguments.
msg227854 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-09-29 22:57
Has the Argument Clinic had an impact on this or is that a different kettle of fish?
History
Date User Action Args
2014-09-29 22:57:00BreamoreBoysetnosy: + BreamoreBoy

messages: + msg227854
versions: + Python 3.5, - Python 3.2, Python 3.3
2013-01-22 12:51:50ezio.melottisetnosy: + ezio.melotti
title: Document lack of support for keyword arguments in C functions -> Document lack of support for keyword arguments in C functions

versions: + Python 3.3, Python 3.4, - Python 3.1
2011-01-02 08:25:21loewissetnosy: loewis, georg.brandl, rhettinger, terry.reedy, pitrou, eric.smith, eric.araujo, twhitema, schof, l0nwlf, docs@python, lukasz.langa
messages: + msg125034
title: Document lack of support for keyword arguments in C functions -> Document lack of support for keyword arguments in C functions
2011-01-02 02:22:55terry.reedysetnosy: loewis, georg.brandl, rhettinger, terry.reedy, pitrou, eric.smith, eric.araujo, twhitema, schof, l0nwlf, docs@python, lukasz.langa
messages: + msg125026
2011-01-02 01:52:48terry.reedysetnosy: + terry.reedy

messages: + msg125024
versions: - Python 2.6
2011-01-02 00:51:16eric.araujolinkissue10789 dependencies
2011-01-02 00:08:55rhettingersetnosy: + rhettinger
messages: + msg125021
2011-01-02 00:03:43georg.brandlsetnosy: loewis, georg.brandl, pitrou, eric.smith, eric.araujo, twhitema, schof, l0nwlf, docs@python, lukasz.langa
messages: + msg125019
2011-01-02 00:01:10pitrousetnosy: loewis, georg.brandl, pitrou, eric.smith, eric.araujo, twhitema, schof, l0nwlf, docs@python, lukasz.langa
messages: + msg125018
2011-01-01 23:59:43georg.brandlsetnosy: loewis, georg.brandl, pitrou, eric.smith, eric.araujo, twhitema, schof, l0nwlf, docs@python, lukasz.langa
messages: + msg125017
2011-01-01 17:52:19eric.araujolinkissue8626 superseder
2010-07-30 13:51:09eric.smithsetnosy: + eric.smith
2010-07-30 13:43:11eric.araujosetmessages: + msg112079
2010-07-30 13:39:46lukasz.langasetmessages: + msg112078
2010-07-30 13:38:34eric.araujosettitle: Explicitly state lack of support for keyword arguments in built-in functions -> Document lack of support for keyword arguments in C functions
2010-07-30 13:38:10eric.araujosetresolution: accepted
stage: needs patch
type: behavior
title: os.mkdir doc comment is incorrect -> Explicitly state lack of support for keyword arguments in built-in functions
versions: + Python 3.2
2010-07-30 13:37:15eric.araujolinkissue9426 superseder
2010-07-30 12:41:31lukasz.langasetnosy: + lukasz.langa
messages: + msg112071
2010-06-13 21:46:52eric.araujosetassignee: georg.brandl -> docs@python

nosy: + docs@python
2010-05-09 03:15:12schofsetmessages: + msg105367
2010-05-09 03:08:13schofsetnosy: + schof
messages: + msg105366
2010-04-24 18:23:23eric.araujosetmessages: + msg104116
2010-04-24 18:16:12pitrousetmessages: + msg104114
2010-04-24 18:12:14eric.araujosetmessages: + msg104113
2010-04-09 12:47:36eric.araujosetmessages: + msg102717
2010-04-09 12:40:26l0nwlfsetmessages: + msg102716
2010-04-09 12:35:43eric.araujosetnosy: + eric.araujo
messages: + msg102714
2010-04-09 12:34:17pitrousetnosy: + loewis, pitrou
messages: + msg102713
2010-04-09 06:21:19l0nwlfsetfiles: + posixmodule.patch
versions: + Python 2.7
nosy: + l0nwlf

messages: + msg102683

keywords: + patch
2010-04-08 22:02:36twhitemacreate