classification
Title: Daily/weekly ABI scan?
Type: enhancement Stage:
Components: Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Christian.Tismer, aponomarenko, benjamin.peterson, eric.snow, lukasz.langa, ncoghlan, ned.deily, petr.viktorin, serhiy.storchaka, steve.dower, vstinner
Priority: normal Keywords:

Created on 2014-04-03 08:50 by ncoghlan, last changed 2019-10-29 09:51 by BTaskaya.

Messages (6)
msg215432 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-04-03 08:50
At Anatoly's prompting, Andrey Ponomarenko set up ABI/API compatibility checks for CPython on the Linux upstream tracker (http://upstream-tracker.org/)

Everything: http://upstream-tracker.org/versions/python.html
Public API/ABI (no leading underscores): http://upstream-tracker.org/versions/python_public_api.html
3.2 limited API/stable ABI (no leading underscores): http://upstream-tracker.org/versions/python_stable_api.html

Andrey's analysis in the python-ideas thread [1] indicates we've likely made some mistakes in exposing struct internals, as well as exposing new APIs unintentionally under older Py_LIMITED_API definitions.

There's probably not too much (if anything) we can do to rectify the past mistakes, but we could set up a daily or weekly check (akin to the existing refleak scan) that monitored for such mistakes to prevent the introduction of any new errors along these lines.

[1] https://mail.python.org/pipermail/python-dev/2014-April/133754.html
msg215441 - (view) Author: Andrey (aponomarenko) Date: 2014-04-03 10:41
There are just three simple steps to setup ABI checker:

1. Create ABI dump of the _reference_ version (see instructions [1]) and commit it to the source tree (ABI-ref.dump):

$> abi-compliance-checker -l python -dump descriptor.xml -dump-path ABI-ref.dump

2. Create ABI dump of the _current_ version (ABI-cur.dump)
3. Compare these ABI dumps:

$> abi-compliance-checker -l python -old ABI-ref.dump -new ABI-cur.dump -report-path abi-report.html

To skip analysis of symbols with leading underscore pass this additional option to the tool when comparing ABI dumps: --skip-internal="\A_"

For more information see [2].

[1] https://mail.python.org/pipermail/python-dev/2014-April/133754.html
[2] http://ispras.linuxbase.org/index.php/ABI_compliance_checker
msg318486 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-06-02 03:43
Updated link to the ABI tracker report: https://abi-laboratory.pro/tracker/timeline/python/

(The stable ABI report still has a separate entry on http://upstream.rosalinux.ru/ but it links to the same report as I've linked above)

I'm bumping this based on the 3.7.0 stable ABI break that Christian Tismer reported for PyIndex_Check - it's clear that we *need* some form of automated testing for the stable ABI, or we're going to continue to see issues like this crop up at release time.

(Folks freshly added to the nosy list: I added you either as release managers for 3.7+, or else because we've previously talked about the stable ABI and its maintainability)
msg318655 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2018-06-04 15:01
FTR, the issue of ABI breakage has come up quite a few times on the mailing list.  Most recently:

  https://mail.python.org/pipermail/python-dev/2018-June/153775.html
msg318656 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2018-06-04 15:03
That email thread contains multiple references to scripts that may be used to identify ABI issues, among other good points.  FWIW, when consolidating the C globals last year I added a tool to track the remaining globals:

  Tools/c-globals

It is similar (at least in intent) to any solution we might pursue relative to the stable ABI.
msg355638 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2019-10-29 09:51
Tools/c-globals moved under Tools/c-analyzer
History
Date User Action Args
2019-10-29 09:51:21BTaskayasetnosy: + BTaskaya
messages: + msg355638
2018-06-04 15:03:21eric.snowsetmessages: + msg318656
2018-06-04 15:01:08eric.snowsetmessages: + msg318655
2018-06-02 04:19:31serhiy.storchakasetnosy: + serhiy.storchaka
2018-06-02 03:43:53ncoghlansetnosy: + vstinner, benjamin.peterson, ned.deily, petr.viktorin, lukasz.langa, eric.snow, steve.dower, Christian.Tismer
messages: + msg318486
2014-04-03 10:41:56aponomarenkosetnosy: + aponomarenko
messages: + msg215441
2014-04-03 08:50:16ncoghlancreate