classification
Title: Scanner class in re module undocumented
Type: Stage:
Components: Documentation Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: asmodai, benjamin.peterson, effbot, georg.brandl, michael.foord, mrabarnett, rhettinger
Priority: low Keywords:

Created on 2009-02-21 16:56 by michael.foord, last changed 2009-05-20 01:31 by rhettinger. This issue is now closed.

Messages (13)
msg82569 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-02-21 16:56
There is a useful Scanner class in the re module which is undocumented.

See:

http://mail.python.org/pipermail/python-dev/2003-April/035075.html
http://www.evanfosmark.com/2009/02/sexy-lexing-with-python/
msg82658 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-02-24 04:57
Any opinions on whether this was intended to be exposed?
msg84326 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-03-28 19:15
The class is commented as being "experimental", and the interface
probably could use improvement (it's a bit awkward to subclass Scanner),
but since it's been around for so long, I guess enough people will be
using it that we have to keep it that way and document it.

Opinions?
msg84435 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-03-29 21:59
I'm sure people are already rely on the intimate details of this class,
so why not?
msg84447 - (view) Author: Matthew Barnett (mrabarnett) * (Python triager) Date: 2009-03-30 00:55
FYI, I did tidy up the class and add a 'scaniter' method when I was
working on issue #2636; it might yet see the light of day if it gets the
go ahead!
msg84907 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-03-31 21:11
OK, so we'll wait for that.
msg84914 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-03-31 21:19
Whether this should be exposed is up to effbot.  It's his code.  He
knows its limitations and he made the original decision to leave it
undocumented.
msg84930 - (view) Author: Matthew Barnett (mrabarnett) * (Python triager) Date: 2009-03-31 22:03
One of the limitations is that it identifies what matched by using
capture groups, so if the expressions provided contain captures then it
gets confused! :-)

I handled that by 1) rejecting named captures and 2) changing unnamed
captures into non-captures.
msg86794 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2009-04-29 10:36
So far effbot hasn't said anything about this. So Frederik, was it
intended to be exposed or not?
msg86795 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-04-29 10:39
Whether it was intended to be exposed or not it is known and used - and
therefore we can't change the API without going through the usual
deprecation process. As it is used and useful it should be documented.
msg86796 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2009-04-29 10:41
OK, clear, then I'll see what I can do.
msg88036 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-05-18 16:44
This is very old code, without a mature API.  Will take a look at
competing recipes to see if this is still the best way of doing this. 
Maybe it should return a generator instead of a list.  Perhaps there
should be some checking for re's that don't combine well.  Also, would
like to see it exercised on a number of common parsing tasks to see how
well it holds up.  Better to do this work than to expose what is there
now and get hit with bug reports, feature requests, deprecations, and
cases where is it almost the right tool for the job.
msg88104 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-05-20 01:31
Looking back at the original thread, good reasons were expresses for not
documenting this code but just leaving it in as an example.
History
Date User Action Args
2009-05-20 01:31:06rhettingersetstatus: open -> closed
resolution: rejected
dependencies: - Adding a new regex module (compatible with re)
messages: + msg88104
2009-05-18 16:44:43rhettingersetpriority: low

messages: + msg88036
versions: + Python 2.7, Python 3.2, - Python 2.6, Python 3.0
2009-04-29 12:53:54rhettingersetassignee: effbot -> rhettinger
2009-04-29 10:41:42asmodaisetmessages: + msg86796
2009-04-29 10:39:39michael.foordsetmessages: + msg86795
2009-04-29 10:36:54asmodaisetnosy: + asmodai
messages: + msg86794
2009-03-31 22:03:52mrabarnettsetmessages: + msg84930
2009-03-31 21:19:46rhettingersetassignee: benjamin.peterson -> effbot
messages: + msg84914
2009-03-31 21:11:02georg.brandlsetdependencies: + Adding a new regex module (compatible with re)
messages: + msg84907
2009-03-30 00:55:33mrabarnettsetnosy: + mrabarnett
messages: + msg84447
2009-03-29 21:59:33benjamin.petersonsetmessages: + msg84435
2009-03-28 19:15:26georg.brandlsetassignee: effbot -> benjamin.peterson

messages: + msg84326
nosy: + benjamin.peterson
2009-02-24 04:57:18rhettingersetassignee: georg.brandl -> effbot
messages: + msg82658
nosy: + rhettinger, effbot
2009-02-21 16:56:04michael.foordcreate