classification
Title: skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: larry Nosy List: larry, llllllllll, serhiy.storchaka, vstinner
Priority: low Keywords: patch

Created on 2015-04-13 02:44 by larry, last changed 2018-07-09 16:05 by llllllllll.

Files
File name Uploaded Description Edit
skipitems.patch llllllllll, 2015-04-13 22:43 review
skipitems.patch llllllllll, 2015-07-15 22:26 review
Pull Requests
URL Status Linked Edit
PR 8204 open llllllllll, 2018-07-09 16:05
Messages (8)
msg240587 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-04-13 02:44
tl;dr: skipitem() in Python 3 still supports "w" and "w#", which were abandoned.  They should be removed.

--

If: 
* you have an extension that parses its arguments with
  PyArg_ParseTupleAndKeywords (or its cousins),
* you have optional positional parameters, and
* you run out of positional arguments

CPython will call skipitem() to skip over the remaining positional parameters so it can process keyword arguments.  It does this by knowing all the format units, and iterating over them and throwing away the various varargs pointers until it hits the keyword arguments part of the format string.

PyArg_ParseTuple() etc. in Python 2 supported "w" and "w#" for parsing "objects with the read-write buffer interface".  These were removed in Python 3.

skipitem() in Python 3 still supports both skipping "w" and "w#".  But in fact the only legal format unit starting with a 'w' in Python 3 is 'w*'.  So a function with a 'w*' as an optional parameter that got skipped would be misinterpreted; skipitem would see the 'w', not recognize the following '*' and leave it there, then the next function that read a character from the format string (either skipitem() or convertsimple()) would see the '*' and throw an "impossible <bad format char>" exception.
msg240794 - (view) Author: Joe Jevnik (llllllllll) * Date: 2015-04-13 22:43
here is a patch and test case.
I added an entry to Misc/NEWS. I tried to follow the format; however, let me know if I have done this incorrectly.
msg246780 - (view) Author: Joe Jevnik (llllllllll) * Date: 2015-07-15 22:26
Sorry it took so long to get back to this. I didn't realize this was still open. I have provided the update to the docs and moved it to the 3.6 section. I also made sure the patch still applies and the tests all pass.
msg247383 - (view) Author: Joe Jevnik (llllllllll) * Date: 2015-07-25 19:36
bumping so that we don't forget about this.
msg253226 - (view) Author: Joe Jevnik (llllllllll) * Date: 2015-10-20 15:52
bumping this issue
msg263709 - (view) Author: Joe Jevnik (llllllllll) * Date: 2016-04-19 05:20
bump
msg263776 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-04-19 21:36
I reviewed skipitems.patch.
msg321300 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-09 06:38
Do you mind to convert your patch into a pull request Joe?
History
Date User Action Args
2018-07-09 16:05:58llllllllllsetpull_requests: + pull_request7754
2018-07-09 06:39:12serhiy.storchakasetcomponents: + Interpreter Core
versions: + Python 3.7, Python 3.8, - Python 3.5
2018-07-09 06:38:38serhiy.storchakasetmessages: + msg321300
2016-04-19 21:36:52vstinnersetnosy: + vstinner
messages: + msg263776
2016-04-19 05:54:10serhiy.storchakasetnosy: + serhiy.storchaka

versions: - Python 3.4
2016-04-19 05:20:46llllllllllsetmessages: + msg263709
2015-10-20 16:02:47berker.peksagsetstage: test needed -> patch review
type: crash -> behavior
versions: + Python 3.6
2015-10-20 15:52:11llllllllllsetmessages: + msg253226
2015-07-25 19:36:41llllllllllsetmessages: + msg247383
2015-07-15 22:26:53llllllllllsetfiles: + skipitems.patch

messages: + msg246780
2015-04-13 22:43:33llllllllllsetfiles: + skipitems.patch

nosy: + llllllllll
messages: + msg240794

keywords: + patch
2015-04-13 02:44:00larrycreate