Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the repr for regular expression match objects #61289

Closed
rhettinger opened this issue Jan 30, 2013 · 24 comments
Closed

Improve the repr for regular expression match objects #61289

rhettinger opened this issue Jan 30, 2013 · 24 comments
Assignees
Labels
stdlib Python modules in the Lib dir topic-regex type-feature A feature request or enhancement

Comments

@rhettinger
Copy link
Contributor

BPO 17087
Nosy @rhettinger, @ezio-melotti, @cjerdonek, @PCManticore, @serhiy-storchaka
Files
  • sre.patch
  • sre_repr2.patch
  • sre_repr3.patch
  • sre_repr4.patch
  • sre_repr5.patch
  • sre_match_repr.patch
  • sre_repr6.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2013-10-20.10:16:20.894>
    created_at = <Date 2013-01-30.23:53:02.426>
    labels = ['expert-regex', 'type-feature', 'library']
    title = 'Improve the repr for regular expression match objects'
    updated_at = <Date 2013-11-25.21:20:38.690>
    user = 'https://github.com/rhettinger'

    bugs.python.org fields:

    activity = <Date 2013-11-25.21:20:38.690>
    actor = 'python-dev'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2013-10-20.10:16:20.894>
    closer = 'serhiy.storchaka'
    components = ['Library (Lib)', 'Regular Expressions']
    creation = <Date 2013-01-30.23:53:02.426>
    creator = 'rhettinger'
    dependencies = []
    files = ['31383', '31744', '31746', '32143', '32144', '32159', '32216']
    hgrepos = []
    issue_num = 17087
    keywords = ['patch']
    message_count = 24.0
    messages = ['180999', '181001', '181002', '181004', '181043', '195687', '197546', '197579', '197609', '197610', '197618', '200042', '200053', '200055', '200059', '200065', '200111', '200157', '200356', '200378', '200549', '200558', '200559', '204412']
    nosy_count = 7.0
    nosy_names = ['rhettinger', 'ezio.melotti', 'mrabarnett', 'chris.jerdonek', 'Claudiu.Popa', 'python-dev', 'serhiy.storchaka']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue17087'
    versions = ['Python 3.4']

    @rhettinger
    Copy link
    Contributor Author

    Experience teaching Python has shown that people have a hard time learning to work with match objects. A contributing cause is the opaque repr:

        >>> import re
        >>> s = 'On 3/14/2013, Python celebrate Pi day.'
        >>> mo = re.search(r'\d+/\d+/\d+', s)
        >>> mo
        <_sre.SRE_Match object at 0x100456100>

    They could explore the match object with dir() and help() and the matchobject methods and attributes:

        >>> dir(mo)
        ['__class__', '__copy__', '__deepcopy__', ...
         'end', 'endpos', 'expand', 'group', ... ]
         
        >>> mo.start()
        3
        >>> mo.end()
        12
        >>> mo.group(0)
        '3/14/2013'

    However, this gets old when experimenting with alternative regular expressions. A better solution is to improve the repr:

        >>> re.search(r'\d+/\d+/\d+', s)
        <SRE Match object: start=3, stop=12, group(0)='3/14/2013'>

    This would make the regular expression module much easier to work with.

    @rhettinger rhettinger added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Jan 30, 2013
    @ezio-melotti
    Copy link
    Member

    Showing start and stop would be OK, but there might be many groups and they might contain lot of text, so they can't simply be included in the repr as they are.
    FWIW there was another issue about changing _sre.SRE_Match to something better, but I can't find it right now.

    @cjerdonek
    Copy link
    Member

    Is this a duplicate of bpo-13592?

    @rhettinger
    Copy link
    Contributor Author

    Just showing group(0) should be helpful. And perhaps the number of groups. If a string is really long, we can truncate it like reprlib does.

    The main goal is to make it easier to work with match objects at the interactive prompt. They are currently too opaque.

    @ezio-melotti
    Copy link
    Member

    bpo-13592 is indeed the issue I was thinking about, but apparently that's about _sre.SRE_Pattern, so it's not the same thing.

    Just showing group(0) should be helpful.

    Often the interesting group is group(1), so showing only group(0) seems a bit arbitrary.

    And perhaps the number of groups.

    If we show only group(0), this might be useful as an indication that there are(n't) other groups.

    If a string is really long, we can truncate it like reprlib does.

    That's certainly an option.

    FWIW I don't usually care about the start/end, and, if included, these values could be included as span=(3,12).

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Aug 20, 2013

    Here's my patch attempt. The repr of a match object has the following format:
    (groups=\d+, span=(start, end), group0=the entire group or the first X characters, where X is represented by a new constant in sre_constants.h, SRE_MATCH_REPR_SIZE).

    @serhiy-storchaka
    Copy link
    Member

    What about such output?

        >>> re.search('p((a)|(b))(c)?', 'unpack')
        <SRE Match object: [2: 5]: 'p'(('a')())('c')>

    Or may be ('p', [['a'], []], ['c']) if you prefer legal Python expression.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Sep 13, 2013

    Serhiy, at the first glance, that repr doesn't make sense to me, thus it seems a little difficult to comprehend.

    @serhiy-storchaka
    Copy link
    Member

    Well, then first will commit a simpler patch. I left comments on Rietveld.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Sep 13, 2013

    Here's the new version. I added a few replies on the Rietveld.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Sep 13, 2013

    Added the new version.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Oct 16, 2013

    Serhiy, are there any left issues with my latest patch? It would be nice if we could get this into 3.4.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Oct 16, 2013

    Added the new patch, which addresses Serhiy's comments.
    Also, this approach fails when bytes are involved:

    >>> import re
    >>> re.search(b"a", b"a")
    Assertion failed: (PyUnicode_Check(op)), function _PyUnicode_CheckConsistency, file Objects/unicodeobject.c, line 309.

    Should a check be added for this also?

    @serhiy-storchaka
    Copy link
    Member

    Use correct first argument to getslice().

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Oct 16, 2013

    Latest patch attached.

    @serhiy-storchaka
    Copy link
    Member

    It is too complicated (and perhaps erroneous). Why not use just self->pattern->logical_charsize?

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Oct 17, 2013

    I could use self->pattern->logical_size, but it seems that I still need the call to getstring for bytes & co, to obtain the view to the underlying buffer (otherwise the group0 part from the repr will contain random bytes). I didn't find a simpler way to achieve this.

    @serhiy-storchaka
    Copy link
    Member

    Well. Here is a patch. I have changed repr() a little. repr() now contains match type qualified name (_sre.SRE_Match). "groups" now equals len(m.groups()). "span" representation now contains a comma (as repr(m.span())).

    Raymond, Ezio, is it good to you?

    @serhiy-storchaka serhiy-storchaka self-assigned this Oct 17, 2013
    @ezio-melotti
    Copy link
    Member

    I discussed this briefly with Serhiy on IRC and I think the repr can be improved.
    Currently it looks like:
    >>> re.compile(r'[/\\]([.]svn)').match('/.svn')
    <_sre.SRE_Match object: groups=1, span=(0, 5), group0='/.svn'>

    One problem is that the group count doesn't include group 0, so from the example repr one would expect that the info are about the 1 (and only) group in "groups=", whereas that is actually group 0 and there's an additional group 1 that is not included in the repr.

    A possible solution is to separate the group count from the info about group 0:
    <_sre.SRE_Match object (1 group); group0='/.svn', span=(0, 5)>

    To make things even less confusing we could avoid calling it group0 and use something like "match=", or alternatively remove the group count (doesn't the count depend only on the regex, and not on the string?).

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Oct 19, 2013

    Added patch based on Serhiy's, which addresses your comments. It drops the group count and renames group0 to match.

    @serhiy-storchaka
    Copy link
    Member

    LGTM (except unrelated empty line at the end of Modules/_sre.c).

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 20, 2013

    New changeset 29764a7bd6ba by Serhiy Storchaka in branch 'default':
    Issue bpo-17087: Improved the repr for regular expression match objects.
    http://hg.python.org/cpython/rev/29764a7bd6ba

    @serhiy-storchaka
    Copy link
    Member

    Thanks all participants for the discussion.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 25, 2013

    New changeset 4ba7a29fe02c by Ezio Melotti in branch 'default':
    bpo-13592, bpo-17087: add whatsnew entry about regex/match object repr improvements.
    http://hg.python.org/cpython/rev/4ba7a29fe02c

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir topic-regex type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants