Title: pygettext doesn't extract docstrings for functions with type annotated params
Type: behavior Stage: patch review
Components: Demos and Tools Versions: Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Tobotimus, miss-islington, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2017-12-05 10:36 by Tobotimus, last changed 2018-02-26 23:23 by miss-islington.

File name Uploaded Description Edit Tobotimus, 2017-12-05 10:36 Input file for pygettext to demonstrate behaviour
pygettext_output.pot Tobotimus, 2017-12-05 10:37 Output file from pygettext for Tobotimus, 2017-12-05 10:47 Demonstrating behaviour with return type annotations
Pull Requests
URL Status Linked Edit
PR 4745 merged Tobotimus, 2017-12-07 09:31
PR 5915 merged miss-islington, 2018-02-26 22:49
PR 5916 merged miss-islington, 2018-02-26 22:50
Messages (5)
msg307652 - (view) Author: Toby Harradine (Tobotimus) * Date: 2017-12-05 10:36
### Expected Behaviour
When running pygettext with the -D CLI flag, all module, class, method and function docstrings should be extracted and outputted to the .pot file.

### Actual Behaviour
In the case of functions whose parameters have PEP 484 type annotations, their docstrings are not being extracted. I have attached two files, one .py file and its corresponding .pot file, as examples of this behaviour.
msg307654 - (view) Author: Toby Harradine (Tobotimus) * Date: 2017-12-05 10:47
Correction for above: the type annotation syntax is actually from PEP 3107, not PEP 484.

I should also point out that this behaviour is occurring for annotated return types of functions and methods as well.

This is occurring on both Windows and Linux.
msg307791 - (view) Author: Toby Harradine (Tobotimus) * Date: 2017-12-07 07:34
Another correction; I said that this is occurring for return type annotations; this is not true. This is only occurring for functions with annotated *arguments*.

The cause of this issue is in pygettext's TokenEater.__suiteseen method. This method eats tokens until it sees a colon operator, after which it assumes a function docstring may follow. In the case of a function with annotated arguments, obviously this is incorrect, as colons would appear before the final colon of the 'suite'.
msg312967 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-02-26 22:48
New changeset eee72d4778a5513038edd5236cdd87ccce2bc60a by Serhiy Storchaka (Tobotimus) in branch 'master':
bpo-32222: Fix pygettext skipping docstrings for funcs with arg typehints (GH-4745)
msg312970 - (view) Author: miss-islington (miss-islington) Date: 2018-02-26 23:23
New changeset 51d95ffc2fb3337dc87277c9af25ecc9a01f991b by Miss Islington (bot) in branch '3.7':
bpo-32222: Fix pygettext skipping docstrings for funcs with arg typehints (GH-4745)
Date User Action Args
2018-02-26 23:23:30miss-islingtonsetnosy: + miss-islington
messages: + msg312970
2018-02-26 22:50:25miss-islingtonsetpull_requests: + pull_request5687
2018-02-26 22:49:27miss-islingtonsetpull_requests: + pull_request5686
2018-02-26 22:48:16serhiy.storchakasetmessages: + msg312967
2017-12-15 12:09:41serhiy.storchakasetassignee: serhiy.storchaka

nosy: + serhiy.storchaka
versions: + Python 3.7, - Python 3.5
2017-12-07 09:31:54Tobotimussetkeywords: + patch
stage: patch review
pull_requests: + pull_request4648
2017-12-07 07:34:51Tobotimussetmessages: + msg307791
2017-12-05 10:47:39Tobotimussetfiles: +

messages: + msg307654
2017-12-05 10:37:30Tobotimussetfiles: + pygettext_output.pot
2017-12-05 10:36:15Tobotimuscreate