classification
Title: Smelly exports (global symbols in python not prefixed with Py or _Py)
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: dmalcolm Nosy List: christian.heimes, dmalcolm, doko, lemburg, loewis, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2008-12-06 01:29 by christian.heimes, last changed 2013-10-12 21:08 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
fix-make-smelly-on-trunk.patch dmalcolm, 2010-01-13 20:18
py3k-add-_Py-prefix-to-avoid-smelliness.patch dmalcolm, 2010-07-20 18:04
Messages (15)
msg77090 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-12-06 01:29
I just found about the smelly build target. It checks for smelly exports.

$ make smelly
[...]
nm -p libpython2.6.a | \
                sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

asdl_int_seq_new
asdl_seq_new
init_ast
init_codecs
initerrno
initgc
initimp
initposix
initpwd
initsignal
init_sre
init_symtable
initthread
initxxsubtype
initzipimport

nm -p libpython3.0.a | \
                sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

_add_one_to_index_C
_add_one_to_index_F
asdl_int_seq_new
asdl_seq_new

These should be checked and fixed.
msg77091 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-06 01:38
Are you suggesting that the init* functions in Python 2.6 should not
longer be exported? Why? And how?
msg77096 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-12-06 02:20
I just suggest that somebody should look into it. I neither know how
critical the issue is nor how to change them. This bug report serves as
a reminder.
msg77292 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-12-08 09:44
For Python 2.x you cannot change the "main" function name of C
extensions, since this is used by the Python import mechanism.
msg77350 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-08 21:56
I guess you *could* change the name of extensions that become builtin
modules. However, I think it is not worth the trouble, especially since
PEP 3121 solves the problem for good, for Python 3.

I propose to simply filter out init[_a-z]+ from the set of "bad" symbols.

That leaves us with asdl_*_seq_new, and _add_one_to_index_; the latter
is already reported as issue3101.
msg77517 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-10 09:05
There is no proposed patch yet, so removing 2.5.3 from the target versions.
msg84408 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-29 14:33
#5591 is a duplicate of this.
msg93793 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-09 14:06
In trunk:

_add_one_to_index_C
_add_one_to_index_F
asdl_int_seq_new
asdl_seq_new
init_ast
init_codecs
initerrno
initgc
initimp
initposix
initpwd
initsignal
init_sre
init_symtable
initthread
initxxsubtype
initzipimport

In py3k:

_add_one_to_index_C
_add_one_to_index_F
asdl_int_seq_new
asdl_seq_new
msg97737 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-01-13 20:18
Re: msg #77350:
> I propose to simply filter out init[_a-z]+ from the set of "bad" 
> symbols.

I'm attaching a patch (to trunk) to Makefile.pre.in which filters out such symbols.

Relevant part of output of "make smelly" on my svn build with this patch:
nm -p libpython2.7.a | \
		sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | \
		grep -v -E "^init[_a-z]+" | sort -u; \

_add_one_to_index_C
_add_one_to_index_F
asdl_int_seq_new
asdl_seq_new
msg110934 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-07-20 17:48
py3k is much cleaner than python 2, due to the change in the module API.

Here's "make smelly"'s output on a recent py3k checkout:
nm -p libpython3.2.a | \
                sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

_add_one_to_index_C
_add_one_to_index_F
asdl_int_seq_new
asdl_seq_new
msg110938 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-07-20 18:04
Patch to py3k which adds the "_Py" prefix to the four listed symbols.

With this, the output from "make smelly" is clean (odorless, perhaps?).

However, adding _Py does seem to go against this comment in Include/asdl.h:
/* It would be nice if the code generated by asdl_c.py was completely
   independent of Python, but it is a goal the requires too much work
   at this stage.  So, for example, I'll represent identifiers as
   interned Python strings.
*/
msg176418 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-11-26 15:15
For Python 3.3 and 3.4 make smelly still lists two exported symbols.

asdl_int_seq_new
asdl_seq_new


Do we have to check the modules, too? Some of them like _ctypes and _decimal export a lot of symbols. Here is a small shell snippet for Makefile.

   for MOD in `find $(srcdir)/build -name '*.s[ol]' ` ; do \
       EXPORT=`nm -p $$MOD | sed -n "/ [TDB] /s/.* //p" | \
               grep -E -v "^_*Py|^_init|^_fini" | sort -u`; \
       if [ -n "$$EXPORT" ]; then \
           echo $$MOD; \
           echo $$EXPORT; \
       fi \
   done
msg176486 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-11-27 19:36
I'd like to focus this issue; it has been open long enough, and deserves to get closed once the original issue is resolved - which was that "make smelly" reports symbols. I think dmalcolm's patch is quite a good start for that.

It may well be that other modules need to be considered - but PLEASE not in this issue. For Unix with shared libraries, the extension modules certainly cause problems, in particular if Python gets embedded in a context (e.g. Apache) that also loads separate copies of the same libraries, and may cause problems if one of our functions collide with some library. However, I see *two* issues falling out of this: one to extend the "make smelly" target to include extension modules, and the other to then fix the extension modules - preferably with one issue per extension module.
msg177551 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-12-15 19:27
This very simple patch should certainly be applied.
msg199622 - (view) Author: Roundup Robot (python-dev) Date: 2013-10-12 20:54
New changeset 142c62a490ce by Antoine Pitrou in branch 'default':
Issue #4555: All exported C symbols are now prefixed with either "Py" or "_Py".
http://hg.python.org/cpython/rev/142c62a490ce
History
Date User Action Args
2013-10-12 21:08:22pitrousetstatus: open -> closed
stage: commit review -> resolved
resolution: fixed
versions: - Python 3.2
2013-10-12 20:54:48python-devsetnosy: + python-dev
messages: + msg199622
2012-12-15 19:27:12pitrousetassignee: dmalcolm
stage: patch review -> commit review
messages: + msg177551
versions: + Python 2.7, - Python 2.6
2012-11-27 19:36:37loewissetmessages: + msg176486
2012-11-26 22:39:01pitrousetstage: test needed -> patch review
2012-11-26 15:15:51christian.heimessetmessages: + msg176418
versions: + Python 3.3, Python 3.4, - Python 3.1, Python 2.7
2010-07-20 18:04:22dmalcolmsetfiles: + py3k-add-_Py-prefix-to-avoid-smelliness.patch

messages: + msg110938
2010-07-20 17:49:23belopolskysettitle: Smelly exports -> Smelly exports (global symbols in python not prefixed with Py or _Py)
2010-07-20 17:48:01dmalcolmsetmessages: + msg110934
2010-01-13 20:18:37dmalcolmsetfiles: + fix-make-smelly-on-trunk.patch

nosy: + dmalcolm
messages: + msg97737

keywords: + patch
2009-10-09 14:06:37pitrousetmessages: + msg93793
versions: + Python 3.2, - Python 3.0
2009-03-29 14:33:26pitrousetnosy: + pitrou, doko
messages: + msg84408
2009-03-29 14:33:00pitroulinkissue5591 superseder
2008-12-10 09:05:29loewissetmessages: + msg77517
versions: - Python 2.5.3
2008-12-08 21:56:16loewissetmessages: + msg77350
2008-12-08 09:44:43lemburgsetnosy: + lemburg
messages: + msg77292
2008-12-06 02:55:30loewissetpriority: high -> normal
2008-12-06 02:20:16christian.heimessetmessages: + msg77096
2008-12-06 01:38:30loewissetnosy: + loewis
messages: + msg77091
2008-12-06 01:29:10christian.heimescreate