classification
Title: textwrap.dedent mishandles empty lines
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.8, Python 3.7, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Julian, docs@python, emilyemorehouse, georg.brandl, gregory.p.smith, gvanrossum, martin.panter, miss-islington, ncoghlan, pitrou, serhiy.storchaka, syadlapalli, terry.reedy
Priority: normal Keywords: easy, patch

Created on 2017-06-25 15:25 by Julian, last changed 2019-06-30 04:41 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 14469 merged syadlapalli, 2019-06-29 22:18
PR 14475 merged miss-islington, 2019-06-30 04:20
PR 14476 merged miss-islington, 2019-06-30 04:21
PR 14477 merged miss-islington, 2019-06-30 04:21
Messages (13)
msg296825 - (view) Author: Julian Berman (Julian) * Date: 2017-06-25 15:25
⊙  python2 -c 'from textwrap import dedent; print repr(dedent(" " * 2 + "\n" + " " * 4 + "\n"))'
'\n\n'

instead of the presumed '\n  \n'

The same appears to be the case for py3.6.


(At first glance, this seems unrelated to http://bugs.python.org/issue19479 although that issue seems to have an unreviewed patch that changes the implementation, so it might also "accidentally" fix the issue).
msg296961 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-06-26 22:02
Some people like to avoid indented blank lines, treating them the same as trailing whitespace. I suspect this behaviour may be intentional.
msg296983 - (view) Author: Emily Morehouse (emilyemorehouse) * (Python committer) Date: 2017-06-27 01:27
I concur with Martin, whitespace is not expected to be preserved on whitespace-only lines nor even considered when finding common leading whitepace. To illustrate this, see the examples below:

The following yields the same (expected) results:

    > python2 -c 'from textwrap import dedent; print repr(dedent(" " * 2 + "\n" + " " * 4 + "\t\n"))'
    > '\n\n'

And a simplified version of a test case from test_textwrap.py:

    > python2 -c 'from textwrap import dedent; print repr(dedent("  Foo\n \n"))'
    > 'Foo\n\n'
msg296997 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-27 04:51
This is related to #19479 at least in the sense of addressing the same module, and being about   It has a patch, so it would be good to accept or reject. Martin, could you take a look?  There is currently no expert listed for textwrap.  Georg is current dormant as far as cpython is concerned.

To call the current behavior a bug, it should conflict with somethings in the docs.  Otherwise, it may be an enhancement proposal, in which case python-ideas would be a good place to discuss.  Julian, look in the docs to see if you find a conflict.
msg297002 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-27 05:13
I think the current behavior is useful, but undocumented. Otherwise a blank line starting with unmatched whitespaces (e.g. "  foo\n\t\n  bar") would block deindenting.

This also matches the behavior of Python parser which ignores all whitespaces in blank lines.

>>> exec('if 1:\n\tpass\n  pass\n')                                                                                                                                                         
Traceback (most recent call last):                                                                                                                                                              
  File "<stdin>", line 1, in <module>                                                                                                                                                           
  File "<string>", line 3                                                                                                                                                                       
    pass                                                                                                                                                                                        
       ^                                                                                                                                                                                        
IndentationError: unindent does not match any outer indentation level                                                                                                                           
>>> exec('if 1:\n\t\n  pass\n')                                                                                                                                                             
>>> 

Added Antoine and Nick as yet two experts of the textwrap module.
msg297007 - (view) Author: Julian Berman (Julian) * Date: 2017-06-27 05:40
@Terry IMHO it conflicts with the fundamental description of the function.

> Remove any common leading whitespace from every line in text.

If this behavior is intentional, textwrap.dedent does not do that, it does that, but also some other stuff.

As for whether it *should* do that, I can't really see how it'd be a good idea to do this -- whether indented blank lines are like trailing whitespace or not is a good reason not to *add* them, but if you've got some already, textwrap.dedent isn't responsible for them, they were there already.

It is also extremely hard to get the other behavior given a function that does this one, but trivial to do the reverse.

Also, the #1 reason to use textwrap.dedent (and also the very next line of the docstring after the one I quoted) is to line up sections of source code. With this behavior, that usecase goes away any time you need trailing whitespace, which is exactly how I found this behavior, by unittesting a function that was returning:

textwrap.dedent(
    """
    foo '
        bar
        baz
    

        quux
     '
    """,
)
msg343988 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-05-30 17:55
The current behavior is desired.  We just need to document it better in https://docs.python.org/3/library/textwrap.html#textwrap.dedent
msg345411 - (view) Author: sushma (syadlapalli) * Date: 2019-06-12 21:32
I'm going to try and contribute this documentation fix.
msg345420 - (view) Author: Julian Berman (Julian) * Date: 2019-06-12 22:15
I still disagree :) but docs are better than nothing.

On Wed, Jun 12, 2019, 18:05 Guido van Rossum <report@bugs.python.org> wrote:

>
> Change by Guido van Rossum <guido@python.org>:
>
>
> ----------
> nosy: +gvanrossum
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue30754>
> _______________________________________
>
msg346909 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-06-30 04:20
New changeset eb97b9211e7c99841d6cae8c63893b3525d5a401 by Gregory P. Smith (tmblweed) in branch 'master':
bpo-30754: Document textwrap.dedent blank line behavior. (GH-14469)
https://github.com/python/cpython/commit/eb97b9211e7c99841d6cae8c63893b3525d5a401
msg346911 - (view) Author: miss-islington (miss-islington) Date: 2019-06-30 04:38
New changeset e2e41cd114ae761fbfee4e7c6539f5df5c8c7116 by Miss Islington (bot) in branch '3.7':
bpo-30754: Document textwrap.dedent blank line behavior. (GH-14469)
https://github.com/python/cpython/commit/e2e41cd114ae761fbfee4e7c6539f5df5c8c7116
msg346912 - (view) Author: miss-islington (miss-islington) Date: 2019-06-30 04:40
New changeset 3e133c401a51f08404b68f11d921f0b406283741 by Miss Islington (bot) in branch '3.8':
bpo-30754: Document textwrap.dedent blank line behavior. (GH-14469)
https://github.com/python/cpython/commit/3e133c401a51f08404b68f11d921f0b406283741
msg346913 - (view) Author: miss-islington (miss-islington) Date: 2019-06-30 04:41
New changeset bc60c47169d1cb33f6fbe1ed64c09a536e82e1c3 by Miss Islington (bot) in branch '2.7':
[2.7] bpo-30754: Document textwrap.dedent blank line behavior. (GH-14469) (GH-14475)
https://github.com/python/cpython/commit/bc60c47169d1cb33f6fbe1ed64c09a536e82e1c3
History
Date User Action Args
2019-06-30 04:41:58miss-islingtonsetmessages: + msg346913
2019-06-30 04:40:44miss-islingtonsetmessages: + msg346912
2019-06-30 04:38:14miss-islingtonsetnosy: + miss-islington
messages: + msg346911
2019-06-30 04:27:50gregory.p.smithsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-06-30 04:21:24miss-islingtonsetpull_requests: + pull_request14294
2019-06-30 04:21:17miss-islingtonsetpull_requests: + pull_request14293
2019-06-30 04:20:24gregory.p.smithsetmessages: + msg346909
2019-06-30 04:20:21miss-islingtonsetpull_requests: + pull_request14292
2019-06-29 22:18:45syadlapallisetkeywords: + patch
stage: patch review
pull_requests: + pull_request14286
2019-06-12 22:15:06Juliansetmessages: + msg345420
2019-06-12 22:05:40gvanrossumsetnosy: + gvanrossum
2019-06-12 21:32:45syadlapallisetnosy: + syadlapalli
messages: + msg345411
2019-05-30 17:55:14gregory.p.smithsetassignee: docs@python
components: + Documentation, - Library (Lib)
versions: + Python 3.8, - Python 3.5, Python 3.6
keywords: + easy
nosy: + gregory.p.smith, docs@python

messages: + msg343988
2017-06-27 05:40:21Juliansetmessages: + msg297007
2017-06-27 05:13:45serhiy.storchakasetnosy: + pitrou, serhiy.storchaka, ncoghlan
messages: + msg297002
2017-06-27 04:51:01terry.reedysetmessages: + msg296997
2017-06-27 01:27:51emilyemorehousesetnosy: + emilyemorehouse
messages: + msg296983
2017-06-26 22:02:56martin.pantersetnosy: + martin.panter
messages: + msg296961
2017-06-25 15:25:24Juliancreate