classification
Title: PyUnicode_ functions not accessible in Limited API on Windows
Type: behavior Stage: needs patch
Components: Windows Versions: Python 3.4, Python 3.3
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bdirks, christian.heimes, haypo, loewis, python-dev
Priority: high Keywords:

Created on 2013-03-15 22:18 by bdirks, last changed 2014-09-21 16:55 by SilentGhost. This issue is now closed.

Messages (7)
msg184268 - (view) Author: Bill Dirks (bdirks) Date: 2013-03-15 22:18
This issue was discovered in 3.3.0 on Windows. Haven't looked at other versions.
Using the Limited API, the PyUnicode_Xxxx() functions are not accessible. A simple demonstration of the problem:

   HMODULE p3 = ::LoadLibraryA("C:\\Python33\\DLLs\\python3.dll");
   FARPROC f;
   f = GetProcAddress(p3, "Py_IsInitialized"); // works
   f = GetProcAddress(p3, "PyImport_AppendInittab"); // works, and so on
   f = GetProcAddress(p3, "PyUnicode_FromString"); // fails
   f = GetProcAddress(p3, "PyUnicode_FromFormat"); // fails
   f = GetProcAddress(p3, "PyUnicode_FromStringAndSize"); // fails, and so on
   ::FreeLibrary(p3);

After some digging I found that python3.dll refers all of its exports on to python33.dll, and the problem exports all have the form:
  PyUnicode_Xxxx -> python33.PyUnicodeUCS2_Xxxx
but python33.dll does not export any PyUnicodeUCS2_ symbols, confirmed by the Dependency Walker tool.

Any Limited API extension using PyUnicode_ functions will compile and link, but fail at runtime. If I understand the problem, the fix is just to correct the .def file.
msg184275 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-03-15 22:49
The Unicode functions were left out of the limited API precisely because of the UCS2/UCS4 issue. If you have an extension module on a narrow Python build using the limited API, it should work without recompilation on a wide build as well.

Now, with Python 3.3, this issue is gone, so it might be reasonable to add  some of the functions to the limited API. Can you propose a list of functions that you think should be there?
msg184293 - (view) Author: Bill Dirks (bdirks) Date: 2013-03-16 05:57
A great number of PyUnicode functions are already added to the Limited API. That is, the declarations are outside of #ifndef Py_LIMITED_API guards in the header files, and the symbols are included in python3.lib and exported from python3.dll. (in 3.3.0)

Can't those functions link to the functions of the same name in python33.dll?

In other words, for example, PyUnicode_FromString in the python3.dll should link to PyUnicode_FromString in python33.dll, not PyUnicodeUCS2_FromString (which doesn't exist in python33.dll). Right?
msg184325 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-03-16 15:31
Ah, I see. Yes, it seems indeed that the def file needs to be changed.
msg201271 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-25 18:03
Martin, do you want to change the def-file for Python 3.4?
msg207276 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-04 09:06
New changeset ea0aa3e32ab5 by Martin v. Löwis in branch '3.3':
Issue #17432: Drop UCS2 from names of Unicode functions in python3.def.
http://hg.python.org/cpython/rev/ea0aa3e32ab5

New changeset 0ea09c824d9b by Martin v. Löwis in branch 'default':
Merge with 3.3: Issue #17432: Drop UCS2 from names of Unicode functions in python3.def.
http://hg.python.org/cpython/rev/0ea09c824d9b
msg207279 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-01-04 10:33
This is now fixed for 3.3 and 3.4. Sorry it took so long.
History
Date User Action Args
2014-09-21 16:55:29SilentGhostsettitle: ucs2 -> PyUnicode_ functions not accessible in Limited API on Windows
2014-09-21 16:55:15SilentGhostsettitle: PyUnicode_ functions not accessible in Limited API on Windows -> ucs2
2014-01-28 07:44:03loewissetstatus: open -> closed
2014-01-04 10:33:00loewissetmessages: + msg207279
2014-01-04 09:06:41python-devsetnosy: + python-dev
messages: + msg207276
2013-10-25 19:32:55pitrousetstage: needs patch
versions: + Python 3.3
2013-10-25 18:03:54christian.heimessetpriority: normal -> high
versions: + Python 3.4, - Python 3.3
nosy: + christian.heimes

messages: + msg201271
2013-03-16 15:31:50loewissetmessages: + msg184325
2013-03-16 13:35:53hayposetnosy: + haypo
2013-03-16 05:57:35bdirkssetmessages: + msg184293
2013-03-15 22:49:46loewissetnosy: + loewis
messages: + msg184275
2013-03-15 22:18:04bdirkscreate