classification
Title: repr(regex) doesn't include actual regex
Type: enhancement Stage: resolved
Components: Regular Expressions Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: alex, chris.jerdonek, dwt, eric.snow, ezio.melotti, hltbra, mrabarnett, pitrou, python-dev, rhettinger, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2011-12-13 10:42 by dwt, last changed 2013-11-25 21:20 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
issue13592_add_repr_to_regex.patch hltbra, 2012-07-19 22:37 review
issue13592_add_repr_to_regex_v2.patch hltbra, 2012-07-20 14:29 review
issue13592_add_repr_to_regex_v2_1.patch hltbra, 2012-07-20 14:34 review
issue13592_add_repr_to_regex_v3.patch serhiy.storchaka, 2013-11-23 16:39 review
issue13592_add_repr_to_regex_v4.patch serhiy.storchaka, 2013-11-23 18:31 review
Messages (27)
msg149382 - (view) Author: Martin H├Ącker (dwt) Date: 2011-12-13 10:42
When calling repr() on a compiled regex pattern like this:

> import re
> repr(re.compile('foo'))

you don't get the pattern of the regex out of the compiled form. Also all my research has shown no getter to allow this.

I noticed this in my application because I was unable to show good error messages for things involving regexes, which is a shame.

So please add the actual regex to the repr() form of the compiled regex, or alternatively provide a getter / property to get at it.
msg149383 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-12-13 11:04
I'm not sure having the pattern in the repr will make it more readable, since the regex might even be very long.  You can use the .pattern attribute if you want to see the pattern.
msg149395 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-13 15:48
> I'm not sure having the pattern in the repr will make it more readable, 
> since the regex might even be very long. 

Hmm, I think it's a reasonable feature request myself.
Oops, I meant "enhancement", not "feature request" :)
msg149396 - (view) Author: Matthew Barnett (mrabarnett) * Date: 2011-12-13 16:58
In reply to Ezio, the repr of a large string, list, tuple or dict is also long.

The repr of a compiled regex should probably also show the flags, but should it just be the numeric value?
msg149397 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-12-13 17:20
ISTM that .pattern is the one way to do it.
msg149398 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-13 18:08
> ISTM that .pattern is the one way to do it.

To me this is like saying the repr() of functions should not show their
name since .__name__ is the one way to do it. repr() is useful for
debugging and logging, why not make it more useful?
msg149407 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-12-13 20:25
If you change the repr, it should at least eval-able, so be sure to capture the flags and whatnot.
msg149409 - (view) Author: Matthew Barnett (mrabarnett) * Date: 2011-12-13 21:24
Actually, one possibility that occurs to me is to provide the flags within the pattern. The .pattern attribute gives the original pattern, but repr could give the flags in-line at the start of the pattern:

>>> # Assuming Python 3.
>>> r = re.compile("a", re.I)
>>> r.flags
34
>>> r.pattern
'a'
>>> repr(r)
"<_sre.SRE_Pattern '(?i)a'>"

I'm not sure how to make it eval-able, unless you mean something more like:

>>> repr(r)
"re.Regex('(?i)a')"

where re.Regex == re.compile, which would be more meaningful than:

>>> repr(r)
"re.compile('(?i)a')"
msg149562 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-12-15 14:31
If an eval-able re.Regex is used, the flags can be showed as second arg, like:
  re.Regex('a', re.I|re.S)
instead of being added to the pattern as in
  re.Regex('(?is)a')

The repr can be generated with something like
  're.Regex({r.pattern!r}, {r.flags})'.format(r=r)
that currently prints
  re.Regex('abc', 50)
but if #11957 is fixed, the result will look like
  re.Regex('abc', re.I|re.S)
for a regex created with
  r = re.compile('abc', re.I|re.S)
msg149649 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-12-17 00:16
> but if #11957 is fixed, the result will look like
  re.Regex('abc', re.I|re.S)

That is what I would like to see.
msg150108 - (view) Author: Matthew Barnett (mrabarnett) * Date: 2011-12-22 20:49
I'm just adding this to the regex module and I've come up against a possible issue. The regex module supports named lists, which could be very big. Should the entire contents of those lists also be shown in the repr?They would have to be if the repr is to be a eval-able.
msg150112 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-22 21:20
> I'm just adding this to the regex module and I've come up against a
> possible issue. The regex module supports named lists, which could be
> very big. Should the entire contents of those lists also be shown in
> the repr?They would have to be if the repr is to be a eval-able.

I don't see how eval()able repr is a big deal. Most reprs aren't, and I
think a readable and informative representation is the real goal.
msg150113 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-12-22 21:44
"I don't see how eval()able repr is a big deal. Most reprs aren't, ..."

Sometimes, I wonder if we're even talking about the same programming language.  Historically, a good deal of effort has gone into creating evalable reprs, if only because they accurately describe an object and because they teach users how to create similar objects.  

But it only takes one committer who doesn't care about evalable reprs to permanently break the pattern for everyone :-(
msg150115 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2011-12-22 21:57
Raymond, Antoine: I don't see your claims as contradictory, it's definitely true that the Python standardlib has historically tried to keep reprs as being eval-able, I think Antoine's correct that the vast majority of 3rd-party code does not keep with that trend.
msg150116 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-12-22 22:08
> But it only takes one committer who doesn't care about evalable reprs
> to permanently break the pattern for everyone :-(

So 95% of our datatypes were committed by a single person? :)
msg165885 - (view) Author: Hugo Lopes Tavares (hltbra) Date: 2012-07-19 22:37
Hey, I started the patch under `default` branch, and get the following working:

    >>> import re
    >>> re.compile("foo")
    re.compile("foo", re.UNICODE)
    >>> re.compile("foo", re.DOTALL)
    re.compile("foo", re.DOTALL|re.UNICODE)
    >>> re.compile("foo", re.DOTALL|re.MULTILINE)
    re.compile("foo", re.MULTILINE|re.DOTALL|re.UNICODE)
    >>>

Do you have any comments on it?

I want to adapt the patch to make it work with python 2.7 too. Do you think is it worthful?

The attached patch was done after commit 3fbfa61634de.
msg165890 - (view) Author: Matthew Barnett (mrabarnett) * Date: 2012-07-19 23:31
Python 2.7 is the end of the Python 2 line, and it's closed except for security fixes.
msg165891 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-07-19 23:39
2.7 is on extended maintenance for normal bugs, but does not get new features/enhancements. It is too late for 3.3 also.
msg165930 - (view) Author: Hugo Lopes Tavares (hltbra) Date: 2012-07-20 14:29
Thanks for the review ezio.melotti.

He has notice a few things in my patch:
* assertEquals is deprecated; should use assertEqual
* the convention is assertEqual(result, expected), not assertEqual(expected, result)
* it should handle quotes correctly
* some lines were longer than 80 chars
* add tests using inline flags (re.I instead of re.IGNORECASE)

And I realized I was not covering the case where no flags are enabled (byte string, for instance). And I have fixed all this issues.

And now I think this patch would work against py2x and py3k anyway.

Attaching a new patch.
msg165932 - (view) Author: Hugo Lopes Tavares (hltbra) Date: 2012-07-20 14:34
Changed two test names to avoid misunderstanding.
msg168966 - (view) Author: Hugo Lopes Tavares (hltbra) Date: 2012-08-23 23:30
Any news about this patch? Is it going to be merged?

When is next CPython release?
msg181051 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2013-02-01 01:23
See also issue 17087 which is essentially the same issue but for match objects.
msg204049 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-23 16:39
Here is fixed and simplified patch.
msg204072 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-23 18:31
* re.UNICODE omitted for string patterns.
* Long patterns are truncated.
msg204103 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-23 20:45
New changeset 8c00677da6c0 by Serhiy Storchaka in branch 'default':
Issue #13592: Improved the repr for regular expression pattern objects.
http://hg.python.org/cpython/rev/8c00677da6c0
msg204105 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-23 20:49
Thank you Hugo for your contribution. Thank you Thomas and Ezio for your reviews and suggestions.
msg204411 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-25 21:20
New changeset 4ba7a29fe02c by Ezio Melotti in branch 'default':
#13592, #17087: add whatsnew entry about regex/match object repr improvements.
http://hg.python.org/cpython/rev/4ba7a29fe02c
History
Date User Action Args
2013-11-25 21:20:38python-devsetmessages: + msg204411
2013-11-23 20:50:09serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-11-23 20:49:50serhiy.storchakasetmessages: + msg204105
2013-11-23 20:45:26python-devsetnosy: + python-dev
messages: + msg204103
2013-11-23 18:31:14serhiy.storchakasetfiles: + issue13592_add_repr_to_regex_v4.patch

messages: + msg204072
2013-11-23 16:39:27serhiy.storchakasetfiles: + issue13592_add_repr_to_regex_v3.patch

messages: + msg204049
2013-10-27 17:18:47serhiy.storchakasetassignee: serhiy.storchaka

nosy: + serhiy.storchaka
2013-02-01 01:23:39chris.jerdoneksetnosy: + chris.jerdonek
messages: + msg181051
2012-08-28 16:59:01ezio.melottisetstage: needs patch -> patch review
2012-08-23 23:30:05hltbrasetmessages: + msg168966
2012-07-20 14:34:41hltbrasetfiles: + issue13592_add_repr_to_regex_v2_1.patch

messages: + msg165932
2012-07-20 14:29:32hltbrasetfiles: + issue13592_add_repr_to_regex_v2.patch

messages: + msg165930
2012-07-19 23:39:41terry.reedysetmessages: + msg165891
versions: + Python 3.4, - Python 3.3
2012-07-19 23:31:42mrabarnettsetmessages: + msg165890
2012-07-19 22:37:59hltbrasetfiles: + issue13592_add_repr_to_regex.patch

nosy: + hltbra
messages: + msg165885

keywords: + patch
2011-12-22 22:08:28pitrousetmessages: + msg150116
2011-12-22 21:57:50alexsetnosy: + alex
messages: + msg150115
2011-12-22 21:44:28rhettingersetmessages: + msg150113
2011-12-22 21:20:09pitrousetmessages: + msg150112
2011-12-22 20:49:43mrabarnettsetmessages: + msg150108
2011-12-17 00:16:40terry.reedysetnosy: + terry.reedy
messages: + msg149649
2011-12-15 19:08:20eric.snowsetnosy: + eric.snow
2011-12-15 14:31:39ezio.melottisetmessages: + msg149562
2011-12-13 21:24:20mrabarnettsetmessages: + msg149409
2011-12-13 20:25:16rhettingersetmessages: + msg149407
2011-12-13 18:08:14pitrousetmessages: + msg149398
2011-12-13 17:20:20rhettingersetnosy: + rhettinger
messages: + msg149397
2011-12-13 16:58:02mrabarnettsetmessages: + msg149396
2011-12-13 15:50:06ezio.melottisetnosy: + mrabarnett
components: + Regular Expressions
2011-12-13 15:48:08pitrousetstatus: pending -> open

type: behavior -> enhancement
versions: - Python 2.7, Python 3.2
nosy: + pitrou

messages: + msg149395
stage: needs patch
2011-12-13 11:04:27ezio.melottisetstatus: open -> pending
versions: + Python 3.2, Python 3.3
nosy: + ezio.melotti

messages: + msg149383
2011-12-13 10:42:23dwtcreate