classification
Title: Doctest in CI uses python binary built from master causing DeprecationWarnings
Type: Stage: resolved
Components: Build, Documentation Versions: Python 3.8
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, matrixise, mdk, xtreak
Priority: normal Keywords:

Created on 2018-10-30 08:34 by xtreak, last changed 2018-11-03 15:16 by mdk. This issue is now closed.

Messages (8)
msg328886 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 08:34
Currently we run doctest in CI and we use the environment variable PYTHON to get the path of python binary to be used for virtualenv creation. We set PYTHON=../python in .travis.yml at [0] before running tests thus this causes the CI to use development version of the binary. Some of the modules used for doctest have incompatible changes related to importing from collections. The DeprecationWarning related code will be removed in 3.8 thus it will cause the doctest to fail while using the modules in CI. Since I have enabled -W in issue34081 they are visible now.

I propose using stable version of Python like python 3.7 so that when we remove collections import related code in master the doctest doesn't fail on Travis. There are respective PRs for this warnings in the repo but using stable version will fix this. We don't need to test the modules against the master branch of Python for doctest which is currently doing.

Sample warning : https://travis-ci.org/python/cpython/jobs/448195459#L2603

> /home/travis/build/python/cpython/Doc/venv/lib/python3.8/site-packages/babel/localedata.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
>  from collections import MutableMapping

On local machine the .travis.yml is not executed thus it uses PYTHON=python3 in Doc/Makefile and thus this not a problem locally building the docs that uses a stable version of Python.


Adding @mdk for feedback on this.

[0] https://github.com/python/cpython/blob/master/.travis.yml#L164
msg328955 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2018-10-30 22:45
> I propose using stable version of Python like python 3.7

If we do so, we won't be able to write doctest about new features (existing in 3.8, not existing in 3.7), which is worse than having warnings.

I think the right move is to do some PR on remote projects (like babel) to fix the warning, future-proofing them from having a real issue in 3.8.
msg328956 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 23:11
I have an open PR for babel https://github.com/python-babel/babel/pull/608 . The problem is merging them and cutting a new release depends on the maintainers availability. Once it's fixed we have to make sure dependencies of babel like sphinx use the latest version and also make a release if needed. I agree it has to be fixed on those projects but coordinating release across various projects depends on the maintainers availability and shouldn't end up as a blocker for CPython's development here and in future with these type of deprecation warnings. I suggested using a stable version because when ran locally make venv for doctest uses python3 as binary name and on CI it uses python built from master branch that might cause confusion. Also it's not necessarily CPython's responsibility to make sure all our external projects work well on development branch.

We can keep this as an open issue while the warnings are fixed in the projects and also use this for reference when the collections import related code has to be removed.
msg328957 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 23:19
> If we do so, we won't be able to write doctest about new features (existing in 3.8, not existing in 3.7), which is worse than having warnings.

I agree with the same. Maybe I am running doctest wrong locally? I use make venv and make doctest. Sorry for the confusion on this.
msg328958 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2018-10-30 23:29
> Maybe I am running doctest wrong locally?

Dunno. IIRC we had problems running doctest with a Python 3.7 on the 3.8 branch, which make sense.

Anyway, if it works today, the first developper writing doctest about a 3.8 feature will cause a doctest failure so no we can't use 3.7 to test 3.8.
msg328959 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-30 23:37
> Anyway, if it works today, the first developper writing doctest about a 3.8 feature will cause a doctest failure so no we can't use 3.7 to test 3.8.

Agreed that we need to run on the master branch binary for 3.8 only features like PEP 572 which is a language level change. Sorry, I forgot about that scenario. I guess it's then the right way to fix the dependencies upstream so that they work properly on development branch going forward.
msg329195 - (view) Author: St├ęphane Wirtel (matrixise) * (Python committer) Date: 2018-11-03 12:22
I think we could close this issue because the DeprecationWarnings are not related to Python itself but to the dependencies.

We could open an issue about the DeprecationWarnings and the dependency like python-babel (thanks for the issue).

@mdk? Can we close this issue?
msg329200 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2018-11-03 15:16
Yes we can close it. Also -W from sphinx-build won't cause errors from those warnings.

Yes we'll probably suffer a bit if someone push the final step of a deprecation if one of our build dependencies is still using it.
History
Date User Action Args
2018-11-03 15:16:40mdksetstatus: open -> closed
resolution: third party
messages: + msg329200

stage: resolved
2018-11-03 12:22:28matrixisesetmessages: + msg329195
2018-10-30 23:37:20xtreaksetmessages: + msg328959
2018-10-30 23:29:27mdksetmessages: + msg328958
2018-10-30 23:19:54xtreaksetmessages: + msg328957
2018-10-30 23:11:47xtreaksetmessages: + msg328956
2018-10-30 22:45:46mdksetmessages: + msg328955
2018-10-30 22:36:37mdksetnosy: + matrixise
2018-10-30 08:34:25xtreakcreate