classification
Title: Use __attribute__((deprecated)) to warn usage of deprecated functions and macros
Type: enhancement Stage: patch review
Components: Build, Interpreter Core Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Michael.Felt, eryksun, ezio.melotti, pitrou, python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2013-11-13 13:11 by vstinner, last changed 2017-05-18 17:56 by serhiy.storchaka.

Files
File name Uploaded Description Edit
mark-deprecated-functions.patch serhiy.storchaka, 2016-11-05 12:05 review
mark-deprecated-functions-2.patch serhiy.storchaka, 2016-11-06 10:49 review
Messages (20)
msg202740 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-13 13:11
Python C API evolves. For example, the Unicode type has been rewriten for scratch to use a more efficient design. Would it be possible to mark deprecated functions as deprecated, so users will be noticed that the API evolved and Python has better functions?

For example, PyUnicode_GET_SIZE() is deprecated and returns a different value than PyUnicode_GET_LENGTH() if wchar_t size is 16-bit (ex: Windows and AIX).

GCC has an nice __attribute__((deprecated)) to tag functions.
msg202743 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-13 13:23
I just commited a change to avoid PyUnicode_GET_SIZE(): this function doesn't handle errors very well, if PyUnicode_AsUnicode() fails, the result is zero. The caller is unable to know that an error occurred.

http://hg.python.org/cpython/rev/28f71af02b69
"""
Don't use deprecated function PyUnicode_GET_SIZE()

Replace it with PyUnicode_GET_LENGTH() or PyUnicode_AsUnicodeAndSize()
"""
msg227674 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-27 14:06
Idea looks good to me.
msg227676 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-27 14:19
Does the __attribute__ work for macros as well?
msg227695 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-27 16:16
Deprecated macros can be replaced by deprecated functions.
msg227727 - (view) Author: Eryk Sun (eryksun) * Date: 2014-09-27 20:51
MSC has __declspec(deprecated). See http://stackoverflow.com/a/21265197, for example. It's a level 3 (/W3) warning.

http://msdn.microsoft.com/en-us/library/ttcz0bys%28v=vs.100%29.aspx
msg227822 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-09-29 20:04
There is already Py_DEPRECATED in Include/pyport.h:


#if defined(__GNUC__) && ((__GNUC__ >= 4) || \
              (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1))
#define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
#else
#define Py_DEPRECATED(VERSION_UNUSED)
#endif


You can add this before '#else':
#elif defined(_MSC_VER) && _MSC_VER >= 1300
#define Py_DEPRECATED(VERSION_UNUSED) __declspec(deprecated)


Py_DEPRECATED is not used since Python 3.0 (it is still used in 2.7).
msg277248 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-23 08:09
Ping myself!

I just cited this GCC attribute in a recent discussion on deprecating macros in the C API:
https://mail.python.org/pipermail/python-dev/2016-September/146537.html
msg280102 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-05 05:39
FYI I used Py_DEPRECATED for marking PyUnicode_AsDecodedObject, PyUnicode_AsDecodedUnicode, PyUnicode_AsEncodedObject and PyUnicode_AsEncodedUnicode.
msg280110 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-05 12:05
Proposed patch marks most deprecated functions. Code is rewritten for using non-deprecated functions if possible. Unfortunately some deprecated function still are used in the code and can't be easier replaced. They are left not marked.

* PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL.

* Py_UNICODE (currently an alias of wchar_t) is used in a number of deprecated functions and bridges between deprecated and new APIs. Maybe it can be just replaced with wchar_t.

* Macros PyUnicode_GET_SIZE, PyUnicode_GET_DATA_SIZE, PyUnicode_AS_UNICODE, PyUnicode_AS_DATA, functions PyUnicode_AsUnicode and PyUnicode_AsUnicodeAndSize are used in a number of places. They can't be easily replaced with wchar-based functions since they return a borrowed reference to cached representation.

* PyUnicode_FromUnicode, PyUnicode_EncodeDecimal and PyUnicode_TransformDecimalToASCII are used only in Modules/_testcapimodule.c. I think we should write tests for modern APIs and eliminate tests for deprecated APIs. Or temporary silence compiler warning in test functions.

* _PyUnicode_ToLowercase, _PyUnicode_ToUppercase and corresponding public macros Py_UNICODE_TOLOWER and Py_UNICODE_TOUPPER are used in Modules/_sre.c (this is a bug in regex implementation). The problem is that more modern functions _PyUnicode_ToLowerFull and _PyUnicode_ToUpperFull is a private API.

All these cases needs separate issues.
msg280134 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-06 10:49
Updated documentation. This part should be backported to all maintained 3.x versions.
msg280150 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2016-11-06 16:52
GCC supports pragmas to locally control warnings.
https://gcc.gnu.org/onlinedocs/gcc-6.2.0/gcc/Diagnostic-Pragmas.html

Example:
====================================
__attribute__((__deprecated__)) int some_deprecated_function(void)
{
  return 0;
};

void some_function(void)
{
  int x, y;
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
  x = some_deprecated_function();
#pragma GCC diagnostic pop
  y = x + 1;
} 

int main(int argc, char** argv)
{
  return 0;
}
====================================
In the above example, call to some_deprecated_function() does not trigger deprecation warning.


'#pragma GCC diagnostic push' and '#pragma GCC diagnostic pop' are supported since GCC 4.6.0.
https://gcc.gnu.org/gcc-4.6/changes.html

'#pragma GCC diagnostic ignored' is documented in documentation of GCC since 4.2.0.


Clang supposedly supports both '#pragma GCC diagnostic ...' and '#pragma clang diagnostic ...':
http://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas
msg280154 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2016-11-06 17:21
_Pragma syntax would be more suitable since it could be used with macros:
https://gcc.gnu.org/onlinedocs/gcc-6.2.0/cpp/Pragmas.html


Then Include/pyport.h (which defines Py_DEPRECATED) could also define:

#if defined(__GNUC__) && ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ >= 6))
#define Py_COMPILER_DIAGNOSTIC_PUSH _Pragma("GCC diagnostic push")
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS _Pragma("GCC diagnostic ignored \"-Wdeprecated-declarations\"")
#define Py_COMPILER_DIAGNOSTIC_POP _Pragma("GCC diagnostic pop")
#else
#define Py_COMPILER_DIAGNOSTIC_PUSH
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS
#define Py_COMPILER_DIAGNOSTIC_POP
#endif


These macros would be used in this way:
void some_function(void)
{
  int x, y;
Py_COMPILER_DIAGNOSTIC_PUSH
Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS
  x = some_deprecated_function();
Py_COMPILER_DIAGNOSTIC_POP
  y = x + 1;
}
msg280192 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-11-07 08:40
I am not compiler specialist, but as I do not use GCC (it adds extra run-time environment (support) requirements when not in a GNU environment such as Linux - I am just wondering what effect this has (e.g., no deprecated message) when not using GCC.

Or, is this a "statement of direction" that only GCC (syntax) compilers are (going to be) supported?

(FYI: there are OSS projects that only accept GCC, and those tend to be non-portable (imho). The "solution" in those cases is to build an additional run-time environment. Personally, I consider that non-portable as I do not want the role of having to maintain security patches for a "non-native" runtime environment (i.e., the maintenance of the (native) rte is performed by the OS supplier, not by me).
msg280195 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-11-07 09:05
> Proposed patch marks most deprecated functions. Code is rewritten for using non-deprecated functions if possible. Unfortunately some deprecated function still are used in the code and can't be easier replaced. They are left not marked.

Hum, I suggest to first update these functions instead of using compiler tricks to ignore deprecation warnings. Once CPython code base doesn't use deprecated functions anymore, deprecate functions (emit a warning).

> All these cases needs separate issues.

I agree :-)

* PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL.

Does this feature (accept NULL) make sense outside CPython core? If not, add a private function exposing the feature.

> * Py_UNICODE (currently an alias of wchar_t) is used in a number of deprecated functions and bridges between deprecated and new APIs. Maybe it can be just replaced with wchar_t.

Which functions are bridges?

Would it be possible to split these bridges into two functions: a private function which doesn't emit a warning, and a public deprecated function which calls the private function?

I dislike the idea of replacing Py_UNICODE* with wchar_t*.


> * Macros PyUnicode_GET_SIZE, PyUnicode_GET_DATA_SIZE, PyUnicode_AS_UNICODE, PyUnicode_AS_DATA, functions PyUnicode_AsUnicode and PyUnicode_AsUnicodeAndSize are used in a number of places. They can't be easily replaced with wchar-based functions since they return a borrowed reference to cached representation.

We must not used these functions in CPython core, except to develop the "bridges" you mentionned before. Please open a separated issue to discuss how to handle the deprecation of the functions using Py_UNICODE*.


* PyUnicode_FromUnicode, PyUnicode_EncodeDecimal and PyUnicode_TransformDecimalToASCII are used only in Modules/_testcapimodule.c. I think we should write tests for modern APIs and eliminate tests for deprecated APIs. Or temporary silence compiler warning in test functions.

* _PyUnicode_ToLowercase, _PyUnicode_ToUppercase and corresponding public macros Py_UNICODE_TOLOWER and Py_UNICODE_TOUPPER are used in Modules/_sre.c (this is a bug in regex implementation). The problem is that more modern functions _PyUnicode_ToLowerFull and _PyUnicode_ToUpperFull is a private API.

"Py_UCS4 _PyUnicode_ToLowercase(Py_UCS4 ch)" doesn't use Py_UNICODE, what is the issue?

"#define Py_UNICODE_ISDECIMAL(ch) _PyUnicode_IsDecimalDigit(ch)" this macro doesn't use the Py_UNICODE type, so we don't have to deprecate it.

Do you want to deprecate functions like Py_UNICODE_ISDECIMAL(ch)?
msg280221 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-07 18:02
> Or, is this a "statement of direction" that only GCC (syntax) compilers are (going to be) supported?

This is only the first step. Since this feature is compiler specific, we should add the support for every compiler separately. Arfrever suggested the syntax for supporting this on Microsoft compiler. But I don't know how to silence warnings on this compiler. Other compilers can support GCC syntax. We should check and switch on this.

> Which functions are bridges?

All deprecated Py_UNICODE related functions use Py_UNICODE. Most of them are bridges to new APIs. Py_UNICODE also is used in implementation of format codes 'u' and 'Z' of PyArg_Parse*. It also is used in many functions that are bridges to Windows API.

> "Py_UCS4 _PyUnicode_ToLowercase(Py_UCS4 ch)" doesn't use Py_UNICODE, what is the issue?

The problem with this function is not related to Py_UNICODE, but that correct Unicode mapping can return multiple characters.

> Do you want to deprecate functions like Py_UNICODE_ISDECIMAL(ch)?

No. It works correctly.
msg280329 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2016-11-08 18:28
About MSVC compiler:

https://msdn.microsoft.com/en-us/library/044swk7y.aspx
https://msdn.microsoft.com/en-us/library/2c8f766e.aspx
https://msdn.microsoft.com/en-us/library/d9x1s805.aspx

So both:

#pragma warning(push)
#pragma warning(disable: 4996)
/* Some code */
#pragma warning(pop)

And:

__pragma(warning(push))
__pragma(warning(disable: 4996))
/* Some code */
__pragma(warning(pop))

Would generally work, but only the second form is suitable for usage inside macros.


Updated proposition of Py_COMPILER_DIAGNOSTIC_* macros:

#if defined(__GNUC__) && ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ >= 6))
#define Py_COMPILER_DIAGNOSTIC_PUSH _Pragma("GCC diagnostic push")
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS _Pragma("GCC diagnostic ignored \"-Wdeprecated-declarations\"")
#define Py_COMPILER_DIAGNOSTIC_POP _Pragma("GCC diagnostic pop")
#elif defined(_MSC_VER) && _MSC_VER >= 1300
#define Py_COMPILER_DIAGNOSTIC_PUSH __pragma(warning(push))
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS __pragma(warning(disable: 4996))
#define Py_COMPILER_DIAGNOSTIC_POP __pragma(warning(pop))
#else
#define Py_COMPILER_DIAGNOSTIC_PUSH
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS
#define Py_COMPILER_DIAGNOSTIC_POP
#endif
msg281257 - (view) Author: Roundup Robot (python-dev) Date: 2016-11-20 10:17
New changeset 75fe67538905 by Serhiy Storchaka in branch '3.5':
Issue #19569: Suggested more appropriate replacements for deprecated Unicode
https://hg.python.org/cpython/rev/75fe67538905

New changeset f26d3f9a958a by Serhiy Storchaka in branch '3.6':
Issue #19569: Suggested more appropriate replacements for deprecated Unicode
https://hg.python.org/cpython/rev/f26d3f9a958a

New changeset 2ae992bc2def by Serhiy Storchaka in branch 'default':
Issue #19569: Suggested more appropriate replacements for deprecated Unicode
https://hg.python.org/cpython/rev/2ae992bc2def

New changeset f692dafe6797 by Serhiy Storchaka in branch 'default':
Issue #19569: Compiler warnings are now emitted if use most of deprecated
https://hg.python.org/cpython/rev/f692dafe6797
msg281258 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-20 10:23
Arfrever, could you please provide your proposition as a patch?
msg293932 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-18 17:56
I do not have the possibility to test on Windows. Maybe someone wants to test Arfrever's suggestion.
History
Date User Action Args
2017-05-18 17:56:42serhiy.storchakasetassignee: serhiy.storchaka ->
messages: + msg293932
2017-04-01 05:49:27serhiy.storchakasetpull_requests: - pull_request1085
2017-03-31 16:36:36dstufftsetpull_requests: + pull_request1085
2016-11-20 10:23:19serhiy.storchakasetmessages: + msg281258
2016-11-20 10:17:06python-devsetnosy: + python-dev
messages: + msg281257
2016-11-20 09:34:03serhiy.storchakasetassignee: serhiy.storchaka
2016-11-08 18:28:13Arfreversetmessages: + msg280329
2016-11-07 18:02:47serhiy.storchakasetmessages: + msg280221
2016-11-07 09:05:30vstinnersetmessages: + msg280195
2016-11-07 08:40:17Michael.Feltsetnosy: + Michael.Felt
messages: + msg280192
2016-11-06 17:21:38Arfreversetmessages: + msg280154
2016-11-06 16:52:49Arfreversetmessages: + msg280150
2016-11-06 10:49:39serhiy.storchakasetfiles: + mark-deprecated-functions-2.patch

messages: + msg280134
2016-11-05 12:05:13serhiy.storchakasetfiles: + mark-deprecated-functions.patch
keywords: + patch
messages: + msg280110

stage: needs patch -> patch review
2016-11-05 05:39:44serhiy.storchakasetmessages: + msg280102
2016-09-23 09:04:53serhiy.storchakasetcomponents: + Build
versions: + Python 3.7, - Python 3.5
2016-09-23 08:09:40vstinnersetmessages: + msg277248
2015-03-18 13:40:35serhiy.storchakasetstage: needs patch
2014-09-29 20:04:53Arfreversetmessages: + msg227822
2014-09-29 19:55:59Arfreversetnosy: + Arfrever
2014-09-27 20:51:07eryksunsetnosy: + eryksun
messages: + msg227727
2014-09-27 16:16:54serhiy.storchakasetmessages: + msg227695
2014-09-27 14:36:34ezio.melottisetnosy: + ezio.melotti
2014-09-27 14:19:55pitrousetnosy: + pitrou
messages: + msg227676
2014-09-27 14:06:39serhiy.storchakasetversions: + Python 3.5, - Python 3.4
nosy: + serhiy.storchaka

messages: + msg227674

components: + Interpreter Core
type: enhancement
2013-11-13 13:23:04vstinnersetmessages: + msg202743
2013-11-13 13:11:16vstinnercreate