classification
Title: Deprecation warnings for the future async and await keywords in Python 3.6
Type: enhancement Stage: resolved
Components: asyncio, Interpreter Core Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: yselivanov Nosy List: anish.shah, brett.cannon, giampaolo.rodola, marco.buttu, ned.deily, python-dev, yselivanov
Priority: release blocker Keywords: patch

Created on 2016-01-22 19:06 by marco.buttu, last changed 2017-11-08 19:58 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
async_await.patch marco.buttu, 2016-02-12 13:53 Patch and related test review
issue_26182.patch yselivanov, 2016-09-14 22:31 review
Messages (24)
msg258833 - (view) Author: Marco Buttu (marco.buttu) * Date: 2016-01-22 19:06
I saw that async and await will become keywords in Python 3.7 :

https://www.python.org/dev/peps/pep-0492/#deprecation-plans

I enabled the deprecation warnings in Python 3.5.1 and Python 3.6 dev, and I noticed that assigning to async or await does not issue any deprecation warning: 

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.5.1 (default, Jan 21 2016, 19:59:28)
[GCC 4.8.4]

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.6.0a0 (default:4b434a4770a9, Jan 12 2016, 13:01:29)
[GCC 4.8.4]
msg259038 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-27 18:00
If someone wants to try and fix this, I would look at how the warning for the 'with' statement was handled (it will either be in the compiler while generating bytecode or somewhere in the parser, but I'm fairly certain it's the compiler).
msg259394 - (view) Author: Marco Buttu (marco.buttu) * Date: 2016-02-02 14:59
The check for the 'with' statement was handled in the parser (parsetok.c), and in Python 2.5 the warnings were enabled by default. 
I do not know how to check if the deprecation warning are enabled, otherwise I would have tried to fix the problem.
msg259409 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-02 18:10
I'm not quite sure what you mean by "do not know how to check if the deprecation warning are enabled". Do you mean how do you make sure you don't emit a DeprecationWarning if someone hasn't turned warnings on? If that's the case then if you look at some example code and read https://docs.python.org/3/c-api/exceptions.html#issuing-warnings you will notice that when you call the warning-related functions, all you have to do is check if the return value is < 0, then return like there is an exception raised. Otherwise the warnings module handles whether something will be shown.
msg259411 - (view) Author: Anish Shah (anish.shah) * Date: 2016-02-02 18:22
I would like to work on this, if it is okay with Marco?
I look at the history of parsetok.c file and I think I can solve this issue. 

Also, I have a doubt - PEP 492 says that "async and await names will be softly deprecated in CPython 3.5 and 3.6".
What exactly does "softly deprecate" mean? is it just same as throwing a warning?
msg259412 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-02 18:35
I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.
msg259413 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-02-02 18:41
> I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.

I actually thought about emitting DeprecationWarnings in 3.6 for async/await NAME tokens.  I think it's a reasonable thing to do to prepare for 3.7, where they will become proper keywords.
msg259534 - (view) Author: Marco Buttu (marco.buttu) * Date: 2016-02-04 06:38
Thank you Brett, your explanation (msg259409) and the c-api doc are really straightforward. I think I can fix it in a few days, just the time to read the devguide and be confident about the workflow.
msg259887 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-02-08 21:17
Assigning the issue to myself to make sure it won't be forgotten before it's too late.  Anish or Marco, feel free to propose a patch.
msg259888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-08 21:26
Can you please mention the python version in the title?
msg259892 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-08 22:16
Oh thank. I didn't understand if you wanted to change Python 3.6 or 3.7.
msg260097 - (view) Author: Marco Buttu (marco.buttu) * Date: 2016-02-11 11:55
I added the PyErr_WarnEx(PyExc_DeprecationWarning, ...) in Python/ast.c, right below the check for None, True and False as names. Running the following test the message is properly printed:

def test_async(self):
    with self.assertWarnsRegex(DeprecationWarning, "reserved keyword"):
        async = 33

However, the test does not pass, because the DeprecationWarning is not triggered. I am sorry but as a cpython beginner I can not figure out how to solve the problem, so I hope someone else can find the time to do it
msg260118 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-11 17:54
You need to temporarily turn on warnings for it to work. For example:

  with warnings.catch_warnings():
      warnings.simplefilter('always')
      with self.assertWarnsRegex(DeprecationWarning, "reserved keyword"):
        exec('async = 33')

Do notice I used exec() as otherwise the warning will be triggered at import instead of when the test runs.
msg260177 - (view) Author: Marco Buttu (marco.buttu) * Date: 2016-02-12 13:53
Thank you Brett, the problem was the missed exec(). With the patch in attachment the tests pass, but it does not seem to me a good solution. Infact, changing the filter at runtime has no effect: 

$ cat foo.py 
import warnings

warnings.simplefilter("always")
async = 33
await = 33

$ ./python foo.py 
$

Does this happen because, putting the PyErr_WarnEx() in Python/ast.c, the warning is issued before the runtime?

Furthermore, if I set the filter from the CL, then the warning is properly triggered, but the file name and line number are wrong: 

$ ./python -Wd foo.py 
sys:1: DeprecationWarning: 'async' will become a reserved keyword in Python 3.7
sys:1: DeprecationWarning: 'await' will become a reserved keyword in Python 3.7
msg260225 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-02-13 04:19
Because parsing is done before execution you can't flip on warnings during runtime in the file you to be affected.

As for the line number, that's because it's raise in C code that doesn't have a trigger in Python code. Try importing the code and you should get the line number of the import. Otherwise you will have to check if there is some function to specify a syntax warning that lets you set the line number explicitly (I don't think there is).
msg276091 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-09-12 19:05
Ned, would it be OK to commit this patch after b1?

async/await are scheduled to become real keywords in 3.7.  Right now they are only keywords in 'async def' blocks, meaning that it's OK to have a class with 'async' or 'await' attributes.

So this will be a backwards compatibility breaking change in 3.7.  This patch makes Python to emit a warning each time you use async or await as an attribute/variable/etc.
msg276097 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-12 19:33
As long as Brett is also OK with it, it can go in for 360b2.
msg276118 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-09-12 21:01
I'm fine with it being in b2 because IMO the warning really should make it in 3.6 and for stuff like this it's more critical to hit the RC for people's testing than the beta to work out semantic changes.
msg276493 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-09-14 22:31
I had to rewrite the patch to make sure it reports correct position and covers all cases where using async/await should trigger a warning.

Brett, could you please take a look at the patch?
msg276575 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-09-15 16:41
I'm going to commit the patch now (I'm going on vacation tomorrow, and I want to watch the buildbots).
msg276576 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-15 16:50
New changeset 82e6017dc841 by Yury Selivanov in branch '3.6':
Issue #26182: Raise DeprecationWarning for improper use of async/await keywords
https://hg.python.org/cpython/rev/82e6017dc841

New changeset 3f8b75173543 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #26182)
https://hg.python.org/cpython/rev/3f8b75173543
msg276577 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-09-15 16:51
Merged.
msg276585 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-09-15 18:09
Sorry I didn't get around to reviewing; I'm sick.

On Thu, Sep 15, 2016, 09:51 Yury Selivanov <report@bugs.python.org> wrote:

>
> Yury Selivanov added the comment:
>
> Merged.
>
> ----------
> resolution:  -> fixed
> stage: needs patch -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26182>
> _______________________________________
>
msg280355 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-11-08 21:54
New changeset 7a996e826f83 by Yury Selivanov in branch '3.6':
Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
https://hg.python.org/cpython/rev/7a996e826f83

New changeset 7b0e79e7f567 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #26182)
https://hg.python.org/cpython/rev/7b0e79e7f567
History
Date User Action Args
2017-11-08 19:58:50serhiy.storchakasetpull_requests: - pull_request970
2017-03-31 16:36:23dstufftsetpull_requests: + pull_request970
2016-11-08 21:54:46python-devsetmessages: + msg280355
2016-09-15 18:09:47brett.cannonsetmessages: + msg276585
2016-09-15 16:51:13yselivanovsetstatus: open -> closed
resolution: fixed
messages: + msg276577

stage: needs patch -> resolved
2016-09-15 16:50:58python-devsetnosy: + python-dev
messages: + msg276576
2016-09-15 16:41:52yselivanovsetmessages: + msg276575
2016-09-14 22:59:20gvanrossumsetnosy: - gvanrossum
2016-09-14 22:31:43yselivanovsetfiles: + issue_26182.patch

messages: + msg276493
2016-09-12 21:01:21brett.cannonsetmessages: + msg276118
2016-09-12 19:33:08ned.deilysetmessages: + msg276097
2016-09-12 19:05:41yselivanovsetpriority: normal -> release blocker
nosy: + gvanrossum, ned.deily
messages: + msg276091

2016-02-13 04:19:07brett.cannonsetmessages: + msg260225
2016-02-12 17:18:36gvanrossumsetnosy: - gvanrossum
2016-02-12 13:53:24marco.buttusetfiles: + async_await.patch
keywords: + patch
messages: + msg260177
2016-02-11 18:00:22vstinnersetnosy: - vstinner
2016-02-11 17:54:30brett.cannonsetmessages: + msg260118
2016-02-11 11:55:14marco.buttusetmessages: + msg260097
2016-02-08 22:16:20vstinnersetmessages: + msg259892
2016-02-08 21:29:04brett.cannonsettitle: Deprecation warnings for the future async and await keywords -> Deprecation warnings for the future async and await keywords in Python 3.6
2016-02-08 21:26:51vstinnersetmessages: + msg259888
2016-02-08 21:17:21yselivanovsetassignee: yselivanov
stage: needs patch
messages: + msg259887
versions: - Python 3.5
2016-02-04 06:38:24marco.buttusetmessages: + msg259534
2016-02-02 18:43:06pitrousetnosy: - pitrou
2016-02-02 18:41:25yselivanovsetmessages: + msg259413
2016-02-02 18:35:51brett.cannonsetnosy: + gvanrossum
messages: + msg259412
2016-02-02 18:22:04anish.shahsetmessages: + msg259411
2016-02-02 18:10:08brett.cannonsetmessages: + msg259409
2016-02-02 14:59:29marco.buttusetmessages: + msg259394
2016-01-27 18:00:07brett.cannonsetnosy: + brett.cannon
messages: + msg259038
2016-01-27 16:25:16gvanrossumsetnosy: - gvanrossum
2016-01-27 12:19:22anish.shahsetnosy: + anish.shah
2016-01-23 11:25:34SilentGhostsetnosy: + gvanrossum, pitrou, vstinner, giampaolo.rodola, yselivanov
components: + asyncio
2016-01-22 19:06:06marco.buttucreate