This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: textwrap.dedent() fails when largest common whitespace is a substring of smallest leading whitespace
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: ezio.melotti, jayvdb, levkivskyi, pitrou, puppet, python-dev, r.david.murray, rhettinger, robertjli, serhiy.storchaka, yjchen, zach.ware
Priority: high Keywords: easy, patch

Created on 2014-06-22 16:44 by robertjli, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
cb18733ce8f1.diff robertjli, 2014-06-22 18:04 review
34e88a05562f.diff robertjli, 2014-06-22 18:37 review
issue21827.patch serhiy.storchaka, 2015-10-28 11:04 review
Repositories containing patches
https://bitbucket.org/bloomberg/cpython#issue21827
Messages (16)
msg221277 - (view) Author: Robert Li (robertjli) * Date: 2014-06-22 16:44
Failing test case: "  \tboo\n \tghost"

expected:          " \tboo\n\tghost"
returns:           "  \tboo\n \tghost"
msg221281 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-06-22 17:18
Confirmed that the test case fails on 2.7, 3.4, and 3.5
msg221282 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-06-22 17:28
This one isn't hard.  Would you like to make a patch?  If not, I get to it this evening.
msg221283 - (view) Author: YJ Chen (yjchen) Date: 2014-06-22 17:30
Hi Raymond- Rob and I have a patch ready. We are figuring out how to upload/submit it. New to this... :)
msg221287 - (view) Author: Robert Li (robertjli) * Date: 2014-06-22 18:03
YJ and I are adding a patch and an additional test.
msg221292 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-06-22 18:28
Raymond confirmed that the issue exists on 2.7 and 3.4, so we'll keep those in the version list.  Whoever makes the commit will take care of backporting the patch, though.
msg224544 - (view) Author: Daniel Eriksson (puppet) * Date: 2014-08-02 09:12
Tested and it works fine on CentOS 6.4 in 2.7, 3.4 and 3.5
msg224547 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-08-02 10:45
I've got it from here.
msg253593 - (view) Author: John Mark Vandenberg (jayvdb) * Date: 2015-10-28 08:43
Not surprising also occurs on Python 3.2, 3.3, & 3.6.
I'm not sure whether fixes like this are suitable to be merged into 3.2/3.3.(is there a document describing this?)
msg253596 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-10-28 11:04
LGTM. But the implementation can be more efficient. Here is optimized patch.

34e88a05562f.diff:
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*100)' -- 'textwrap.dedent(s)'
10000 loops, best of 3: 87.5 usec per loop
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*1000)' -- 'textwrap.dedent(s)'
1000 loops, best of 3: 780 usec per loop

issue21827.patch:
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*100)' -- 'textwrap.dedent(s)'
10000 loops, best of 3: 51 usec per loop
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*1000)' -- 'textwrap.dedent(s)'
1000 loops, best of 3: 395 usec per loop
msg253600 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-28 14:41
It's one of the PEPs (I forget which one).  We only apply non-security fixes to the most recent non-development branch...except that for now we also apply bug fixes to 2.7.  So, 2.7, 3.5, and 3.6 are the targets for this issue.
msg253605 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-10-28 15:46
Serhiy, your way looks best.  Go ahead with it.
msg253609 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-10-28 17:14
I thought that 3.4 will switch to "security fixes only" mode after releasing first release since releasing 3.5.0. It was so with 3.2 and 3.3.
msg253612 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-28 17:37
You are right, I'd forgotten we hadn't done 3.4 final yet.
msg253620 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-10-28 19:46
New changeset faeeb8dbe432 by Serhiy Storchaka in branch '3.4':
Issue #21827: Fixed textwrap.dedent() for the case when largest common
https://hg.python.org/cpython/rev/faeeb8dbe432

New changeset c307ae7b3d9f by Serhiy Storchaka in branch '2.7':
Issue #21827: Fixed textwrap.dedent() for the case when largest common
https://hg.python.org/cpython/rev/c307ae7b3d9f

New changeset 3f29be82c944 by Serhiy Storchaka in branch '3.5':
Issue #21827: Fixed textwrap.dedent() for the case when largest common
https://hg.python.org/cpython/rev/3f29be82c944

New changeset 8d3932671e48 by Serhiy Storchaka in branch 'default':
Issue #21827: Fixed textwrap.dedent() for the case when largest common
https://hg.python.org/cpython/rev/8d3932671e48
msg253621 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-10-28 19:50
Thank you Robert for your report and patch.
History
Date User Action Args
2022-04-11 14:58:05adminsetgithub: 66026
2015-12-15 17:10:01r.david.murraylinkissue25871 superseder
2015-10-28 19:50:48serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg253621

stage: patch review -> resolved
2015-10-28 19:46:09python-devsetnosy: + python-dev
messages: + msg253620
2015-10-28 17:37:47r.david.murraysetversions: + Python 3.4
2015-10-28 17:37:39r.david.murraysetmessages: + msg253612
2015-10-28 17:14:32serhiy.storchakasetmessages: + msg253609
2015-10-28 15:46:50rhettingersetassignee: rhettinger -> serhiy.storchaka
messages: + msg253605
2015-10-28 14:41:25r.david.murraysetmessages: + msg253600
versions: - Python 3.4
2015-10-28 11:04:14serhiy.storchakasetfiles: + issue21827.patch
versions: - Python 3.2, Python 3.3
nosy: + serhiy.storchaka

messages: + msg253596

stage: needs patch -> patch review
2015-10-28 08:43:16jayvdbsetnosy: + jayvdb

messages: + msg253593
versions: + Python 3.2, Python 3.3, Python 3.6
2015-06-29 12:13:06levkivskyisetnosy: + levkivskyi
2014-08-02 10:45:12rhettingersetmessages: + msg224547
2014-08-02 09:12:29puppetsetnosy: + puppet, ezio.melotti
messages: + msg224544
2014-06-22 18:37:45robertjlisetfiles: + 34e88a05562f.diff
2014-06-22 18:28:23zach.waresetmessages: + msg221292
versions: + Python 2.7, Python 3.4
2014-06-22 18:04:29robertjlisetfiles: + cb18733ce8f1.diff
keywords: + patch
2014-06-22 18:03:54robertjlisethgrepos: + hgrepo258
messages: + msg221287
versions: - Python 2.7, Python 3.4
2014-06-22 17:32:09zach.waresetnosy: + zach.ware
2014-06-22 17:30:48yjchensetmessages: + msg221283
2014-06-22 17:28:00rhettingersetkeywords: + easy

messages: + msg221282
2014-06-22 17:18:26rhettingersetpriority: normal -> high

stage: needs patch
messages: + msg221281
versions: + Python 2.7, Python 3.4
2014-06-22 17:07:15rhettingersetassignee: rhettinger

nosy: + rhettinger
2014-06-22 16:44:47robertjlicreate