classification
Title: Segmentation fault with empty "encodings" subdirectory of directory in PYTHONPATH
Type: crash Stage:
Components: Interpreter Core, Unicode Versions: Python 3.1, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, christian.heimes, techtonik, vstinner
Priority: normal Keywords: patch

Created on 2010-05-19 13:27 by Arfrever, last changed 2012-10-22 23:09 by Arfrever. This issue is now closed.

Files
File name Uploaded Description Edit
warnings_bootstrap.patch vstinner, 2010-05-19 16:46
Messages (9)
msg106066 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-05-19 13:27
Python 3.1 and 3.2 fail with segmentation fault when a directory in PYTHONPATH contains an empty "encodings" subdirectory.

$ cd /tmp 
$ mkdir encodings
$ PYTHONPATH=. python3.1
Segmentation fault
msg106078 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-19 16:46
The problem is that a warning is emited before the _warnings module is initialized: get_filter() uses _filters which is equal to NULL.

Attached patch initialize the _warnings module (but not the warnings module) before initfsencoding(). initfsencoding() is the first function loading modules (especially the encoding package) and so _warnings have to be initialized before.

The patch does also fix get_filter() to avoid the crash. This fix alone is not enough to fix this issue: Python cannot start because PyErr_WarnEx() returns -1 if _warnings is not initialized yet.

I tried to display the warning (Not importing directory '/tmp/encodings': missing __init__.py) but it looks to be impossible: display a warning requires that warnings is loaded, but import warnings emits the warning. warnings imports indirectly encodings (linecache -> tokenize -> codecs). I consider that avoiding the crash is enough :-)

--

The encodings can be not empty: the crash occurs if there is an encoding directory is sys.path without the __init__.py file.


   $ mkdir encodings
   $ touch encodings/a
   $ touch encodings/b
   $ touch encodings/c
   $ PYTHONPATH=$PWD ./python 
   Erreur de segmentation
msg106095 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-19 20:41
Fix commited to py3k (r81364). Wait for the buildbots beforing doing the backport.
msg106190 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-20 21:01
> Fix commited to py3k (r81364).

Ok, buildbots are happy: backported as r81383 in 3.1
msg106193 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-05-20 21:44
> Ok, buildbots are happy: backported as r81383 in 3.1

My test doesn't work on Windows. We cannot delete a directory if it's the 
current working directory: let's try r81384.

py3k commit is simpler because it uses support.test_cwd() ;-)
msg173516 - (view) Author: anatoly techtonik (techtonik) Date: 2012-10-22 13:42
This is still an issue for Windows.
msg173518 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-10-22 13:49
> This is still an issue for Windows.

See my message on python-dev: your issue (with an invalid PYTHONHOME,
Python exits with abort) is not related to this one, and it's not a
bug (abort is not a bug).
msg173520 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-10-22 13:56
I agree with Victor.
msg173538 - (view) Author: anatoly techtonik (techtonik) Date: 2012-10-22 16:22
Oh, sorry. I am just testing Python 3.3 and there are so many issues that hardly can test and fill them all.
History
Date User Action Args
2012-10-22 23:09:09Arfreversetversions: - Python 3.3
2012-10-22 16:22:45techtoniksetmessages: + msg173538
2012-10-22 13:56:49christian.heimessetnosy: + christian.heimes
messages: + msg173520
2012-10-22 13:49:37vstinnersetmessages: + msg173518
2012-10-22 13:42:24techtoniksetnosy: + techtonik

messages: + msg173516
versions: + Python 3.3
2010-05-20 21:44:09vstinnersetmessages: + msg106193
2010-05-20 21:01:50vstinnersetstatus: pending -> closed

messages: + msg106190
2010-05-19 20:41:25vstinnersetstatus: open -> pending
resolution: fixed
messages: + msg106095
2010-05-19 16:46:56vstinnersetfiles: + warnings_bootstrap.patch
keywords: + patch
messages: + msg106078
2010-05-19 13:27:20Arfrevercreate