classification
Title: document PEP 448: unpacking generalization
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Jelle Zijlstra, NeilGirdhar, barry, benjamin.peterson, berker.peksag, docs@python, ezio.melotti, martin.panter, moigagoo, paul.moore, python-dev, r.david.murray, steve.dower, supriyanto maftuh, supriyantomaftuh, terry.reedy, tim.golden, vstinner, zach.ware
Priority: high Keywords: easy, patch

Created on 2015-05-06 13:18 by benjamin.peterson, last changed 2016-06-12 07:17 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
whatsnew.diff NeilGirdhar, 2015-05-14 06:37 review
wn2.diff NeilGirdhar, 2015-05-29 21:01 review
wn2.diff NeilGirdhar, 2015-05-29 21:04 review
reference_calls_syntax_update.diff moigagoo, 2015-07-12 22:20 Update of the function call expression syntax.
replace_sequence_with_iterable.diff moigagoo, 2015-07-12 22:21 Replace "sequence" with "iterable" in regard to the assigned object in the assignment statement.
issue24136-expressions.patch Jelle Zijlstra, 2016-06-08 05:51 revised patch for call syntax in the language reference review
unpacking-doc.patch martin.panter, 2016-06-11 05:25 review
Messages (19)
msg242668 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-05-06 13:18
PEP 448 has been implemented (#2292), but the documentation hasn't been updated. Updating the documentation will improve looking through

Doc/reference/*

and making sure the documentation (and grammar) for calls and assignments is updated for PEP 448's new syntax.

I'm marking this as "easy" because it might be a good first bug.
msg243160 - (view) Author: Neil Girdhar (NeilGirdhar) * Date: 2015-05-14 06:37
Just updated the "what's new".  Also, thank you for adding my name to Misc/Acks.  Should we also add Joshua Landau's name?  He helped me quite a bit with the implementation, and he wrote the PEP.
msg244417 - (view) Author: Neil Girdhar (NeilGirdhar) * Date: 2015-05-29 21:01
Simplified functools.partial documentation.
msg246381 - (view) Author: Konstantin Molchanov (moigagoo) * Date: 2015-07-06 22:25
Hi!

I'd like to update the docs with the examples of the new syntax usage. This is my first contribution to the Python docs, so I'd like to ask for some assistance.

I'm going to start with adding an example to the tutorial (https://docs.python.org/3.5/tutorial/introduction.html#lists). I wanted to demonstrate the new syntax with string too (https://docs.python.org/3.5/tutorial/introduction.html#strings), but it turned out to produce somewhat unexpected results:

>>> s = 'And now'
>>> first, *rest = s
>>> # I expected it to be synonymous
>>> # to ``first, rest = s[0], s[1:]``
>>> # ``first`` is expected to be 'A',
>>> # ``rest`` is expected to be 'nd now'.
>>> # ``first`` is 'A', as expected:
>>> first
'A'
>>> # But ``rest`` is implicitly turned into a list:
>>> rest
['n', 'd', ' ', 'n', 'o', 'w', ' ', 'f', 'o', 'r', ' ', 's', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g', ' ', 'c', 'o', 'm', 'p', 'l', 'e', 't', 'e', 'l', 'y', ' ', 'd', 'i', 'f', 'f', 'e', 'r', 'e', 'n', 't']

Is this behavior intended? Why wasn't ``first`` converted into ['A'] as well? Am I just not supposed to use the new unpacking with strings?

Thanks,
Konstantin
msg246385 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-06 23:59
Yes I think it is expected and documented that the leftovers are turned into a list. See <https://docs.python.org/3.5/reference/simple_stmts.html#index-6>. I originally had similar confusion, expectating the starred target to become a tuple, because people often use tuple-like syntax, but:

>>> generator_expression = (2**i for i in range(4))
>>> (one, *a_list, eight) = generator_expression
>>> a_list  # Not a tuple!
[2, 4]

One thing in the section I linked above that should also be fixed is that the assigned object may be any iterable, not just a sequence.

About changing the tutorial, just be careful you don’t add unnecessary complication too early. The original * and ** syntax for function parameters is not mentioned until <https://docs.python.org/3.5/tutorial/controlflow.html#more-on-defining-functions>. Later, argument unpacking: <https://docs.python.org/3.5/tutorial/controlflow.html#unpacking-argument-lists>. Assignment unpacking doesn’t seem to mentioned at all (not that I am saying it should be). It might be higher priority to update the main reference documentation first.
msg246393 - (view) Author: Konstantin Molchanov (moigagoo) * Date: 2015-07-07 07:17
@vadmium thanks for the assistance! I'll kick off with the reference then.

P.S. Am I the only one who doesn't receive any emails from the tracker? I never got the registration link or a follow-up notification from this issue. Am I missing something?

P.P.S. I'm not yet familiar with the local etiquette, so please forgive me if I'm unintentionally breaking some rules. Is @mentioning OK?
msg246446 - (view) Author: Neil Girdhar (NeilGirdhar) * Date: 2015-07-08 03:25
I don't receive emails from the tracker anymore either and I have no idea why that is.
msg246452 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-08 09:05
FWIW, I still emails from the tracker, even the ones with my own comments and changes. All I can suggest is check the address you have set, check for spam, etc.

I don’t @mentioning will do anything here. But as long as the person is in the nosy list they _should_ get an email (in theory :).
msg246485 - (view) Author: Neil Girdhar (NeilGirdhar) * Date: 2015-07-09 11:16
Copied from closed issue 24240:

Since Grammar/Grammar relies on semantic postprocessing in ast.c,
it would be nice to have an update of the (human readable) Grammar
in the language reference docs.
msg246675 - (view) Author: Konstantin Molchanov (moigagoo) * Date: 2015-07-12 22:20
I've updated the Calls syntax reference in reference/expressions and the assignment object description in reference/simple_stmts.

Please tell me if I'm generally doing OK. If I'm not, please guide me to the right direction.
msg261616 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-03-11 23:42
It is now 10 months and 2 releases since the rather large code patch.  Documenting the extensive changes does not seem easy to me ;-).  Certainly, a beginner needs feedback.
msg263262 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-04-12 16:19
supriyanto maftuh,st, please don't play with tracker items.
msg267790 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-06-08 05:37
Here's what I found reviewing what needs to be done here:

- Neil's What's New patch apparently made it into the 3.5 release notes.
- moigagoo's two patches haven't been committed. The text of both looks ok to me, but the patches don't apply cleanly.

Other areas that still need updates:
- https://docs.python.org/3/reference/expressions.html#list-displays and the sections for set, dict, and tuple displays need to mention PEP 448-style syntax.
msg267792 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-06-08 05:51
This updates reference_calls_syntax_update.diff. The previous patch's grammard had a mistake; it was missing commas between arguments.

I believe all other patches in this diff are now obsolete.
msg267822 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-08 09:43
Thanks for helping with this Jelle.

The documentation of unpacking sequences vs iterables was adjusted in 3.6 as part of Issue 23275. I guess part of revision 8a0754fed986 should be extracted to 3.5 as well.

Looking at the function call syntax, positional and starred should be optional. I don’t think your syntax would allow print(file=stderr).
msg268183 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-11 05:25
Here is a new patch that also updates the documentation for list etc displays as well as function calls. Let me know what you think.

The 3.5 What’s New notes were written separately; Neil’s patch was never applied. But I have rescued his update for functools.partial() in my new patch.

While experimenting with the current behaviour, I found some surprising inconsistencies. The following syntaxes are allowed:

>>> x, *y
>>> a = x, *y
>>> f"{x, *y}"  # New in 3.6
>>> async def f(): await x, *y

But the following all produce “SyntaxError: invalid syntax”:

>>> a += x, *y
>>> eval("x, *y")
>>> def f(): return x, *y
>>> def f(): yield x, *y
>>> for i in x, *y: ...

Also, the expressions allowed for unpacking in general are more limited than in function calls:

>>> f(x, *y == z)  # Allowed
>>> (x, *y == z)
SyntaxError: invalid syntax
msg268290 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-06-11 22:40
Thanks for writing a better patch. The patch looks good to me and it builds correctly.
msg268312 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-12 02:06
New changeset a7e04b4e51b2 by Martin Panter in branch '3.5':
Issue #24136: Document generalized unpacking, PEP 448
https://hg.python.org/cpython/rev/a7e04b4e51b2

New changeset 4cf3389cd8e6 by Martin Panter in branch 'default':
Issue #24136: Merge unpacking doc from 3.5
https://hg.python.org/cpython/rev/4cf3389cd8e6

New changeset 2c10f0e92256 by Martin Panter in branch 'default':
Issue #24136: Adjust f-strings doc for interable unpacking
https://hg.python.org/cpython/rev/2c10f0e92256
msg268337 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-12 07:17
Thanks for the review. I committed my patch in the hope that it makes it into 3.5.2, but if people want to suggest further improvements etc that is okay.
History
Date User Action Args
2016-06-12 07:17:55martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg268337

stage: patch review -> resolved
2016-06-12 02:06:37python-devsetnosy: + python-dev
messages: + msg268312
2016-06-11 22:40:33Jelle Zijlstrasetmessages: + msg268290
2016-06-11 05:25:17martin.pantersetfiles: + unpacking-doc.patch

messages: + msg268183
2016-06-08 09:50:28pitrousetnosy: - pitrou
2016-06-08 09:43:13martin.pantersetmessages: + msg267822
2016-06-08 05:51:14Jelle Zijlstrasetfiles: + issue24136-expressions.patch

messages: + msg267792
2016-06-08 05:37:18Jelle Zijlstrasetnosy: + Jelle Zijlstra
messages: + msg267790
2016-04-12 19:07:42zach.waresethgrepos: - hgrepo336
2016-04-12 17:41:15brett.cannonsetnosy: - brett.cannon
2016-04-12 16:19:28berker.peksagsetmessages: - msg263250
2016-04-12 16:19:03berker.peksagsetmessages: + msg263262
components: - Build, Unicode, Windows, XML, IO, Benchmarks, email
versions: - Python 2.7, Python 3.2, Python 3.3, Python 3.4
2016-04-12 15:34:15supriyantomaftuhsetversions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4
nosy: + supriyanto maftuh, tim.golden, supriyantomaftuh, ezio.melotti, r.david.murray, zach.ware, steve.dower, pitrou, barry, brett.cannon, paul.moore, vstinner

messages: + msg263250

hgrepos: + hgrepo336
components: + Build, Unicode, Windows, XML, IO, Benchmarks, email
2016-03-11 23:50:55terry.reedylinkissue26500 superseder
2016-03-11 23:42:38terry.reedysetnosy: + terry.reedy
messages: + msg261616
2015-07-13 20:59:54berker.peksagsetnosy: + berker.peksag
stage: needs patch -> patch review

versions: + Python 3.6
2015-07-12 22:21:21moigagoosetfiles: + replace_sequence_with_iterable.diff
2015-07-12 22:20:21moigagoosetfiles: + reference_calls_syntax_update.diff

messages: + msg246675
2015-07-09 11:16:24NeilGirdharsetmessages: + msg246485
2015-07-08 09:05:32martin.pantersetmessages: + msg246452
2015-07-08 03:25:38NeilGirdharsetmessages: + msg246446
2015-07-07 07:17:47moigagoosetmessages: + msg246393
2015-07-06 23:59:05martin.pantersetmessages: + msg246385
title: document PEP 448 -> document PEP 448: unpacking generalization
2015-07-06 22:25:16moigagoosetnosy: + moigagoo
messages: + msg246381
2015-05-29 21:04:11NeilGirdharsetfiles: + wn2.diff
2015-05-29 21:01:57NeilGirdharsetfiles: + wn2.diff

messages: + msg244417
2015-05-20 20:12:23benjamin.petersonlinkissue24240 superseder
2015-05-14 06:37:23NeilGirdharsetfiles: + whatsnew.diff

nosy: + NeilGirdhar
messages: + msg243160

keywords: + patch
2015-05-13 07:42:20martin.pantersetnosy: + martin.panter
2015-05-06 13:18:47benjamin.petersoncreate