classification
Title: Include\accu.h incompatible with Windows.h
Type: compile error Stage: resolved
Components: Build, Windows Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, benjamin.peterson, georg.brandl, jeffr@livedata.com, kristjan.jonsson, pitrou, python-dev, skrah
Priority: release blocker Keywords: patch

Created on 2012-03-22 11:53 by jeffr@livedata.com, last changed 2012-04-28 08:22 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
issue14387.diff skrah, 2012-03-22 19:22 review
Messages (28)
msg156546 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-03-22 11:53
Windows.h includes RpcNdr.h which does this:

#define small char

accu.h in Python\Include (Python 3.2.3rc2) has this in it:

typedef struct {
    PyObject *large;  /* A list of previously accumulated large strings */
    PyObject *small;  /* Pending small strings */
} _PyAccu;

An extension which includes both Windows.h and Python.h won't compile due to the Windows.h definition of "small".

The compiler error (VS2008) starts with:

c:\python32\include\accu.h(21) : error C2059: syntax error : 'type'
msg156547 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-03-22 12:19
I suggest to remove accu.h from Python.h: all functions are private anyway, and it's only used by couple of files.
msg156560 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-22 13:48
New changeset f34ac2e9d5cf by Antoine Pitrou in branch '3.2':
Issue #14387: Do not include accu.h from Python.h.
http://hg.python.org/cpython/rev/f34ac2e9d5cf

New changeset 5fe7d19ec49a by Antoine Pitrou in branch 'default':
Issue #14387: Do not include accu.h from Python.h.
http://hg.python.org/cpython/rev/5fe7d19ec49a
msg156561 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-22 13:49
Fixed according to Amaury's suggestion.
Georg, Benjamin, I think this should probably go into the final releases.
msg156588 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-03-22 16:20
Sounds reasonable to me.
msg156589 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-03-22 16:22
Hmm, I still get the error in 13cefcbcc7da:

c:\Users\stefan\pydev\cpython\Include\accu.h(21) : error C2059: syntax error : 'type'               
c:\Users\stefan\pydev\cpython\Include\accu.h(22) : error C2059: syntax error : '}'                  
c:\Users\stefan\pydev\cpython\Include\accu.h(24) : error C2143: syntax error : missing ')' before '*
'                                                                                                   
c:\Users\stefan\pydev\cpython\Include\accu.h(24) : error C2143: syntax error : missing '{' before '*
'                                                                                                   
c:\Users\stefan\pydev\cpython\Include\accu.h(24) : error C2059: syntax error : ')'                  
c:\Users\stefan\pydev\cpython\Include\accu.h(25) : error C2143: syntax error : missing ')' before '*
'                                                                                                   
c:\Users\stefan\pydev\cpython\Include\accu.h(25) : error C2143: syntax error : missing '{' before '*
'                                                                                                   
c:\Users\stefan\pydev\cpython\Include\accu.h(25) : error C2371: 'PyObject' : redefinit




[...]

c:\users\stefan\pydev\cpython\objects\stringlib/unicode_format.h(128) : error C2143: syntax error : 
missing ')' before '*'                                                                              
c:\users\stefan\pydev\cpython\objects\stringlib/unicode_format.h(128) : error C2143: syntax error : 
missing '{' before '*'                                                                              
c:\users\stefan\pydev\cpython\objects\stringlib/unicode_format.h(128) : error C2371: 'PyObject' : re
definition; different basic types
msg156590 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-03-22 16:24
We can of course also rename "small".
msg156596 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-22 17:21
> Hmm, I still get the error in 13cefcbcc7da:

Can you try to find a fix? I don't have anything to test here.
msg156598 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-03-22 17:50
The easiest fix would be to rename the "small" field in accu.h (and of course in any file that references that field.)  I have no idea who uses this file.

Perhaps the 2nd easiest fix would be to put this after the #include of Windows.h

#ifdef small
#undef small
#endif

As long as any extension also including Windows.h included Windows.h BEFORE Python.h, this should fix the problem for an extension builder.

A further thought is to put

#define WIN32_LEAN_AND_MEAN

right before the #include Windows.h

This doesn't fix the extension problem in that every #include of Windows.h would need to do this (SQLITE doesn't right now) but if all Python needs from Windows.h is in the "WIN32_LEAN_AND_MEAN" subset, it seems like a good thing in general, no?
msg156607 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-03-22 19:22
Jeff's #undef suggestion works here (see patch). But I have a question:

Where is Windows.h pulled in anyway? A grep only shows signalmodule.c:

$ find . -name "\.hg" -prune -o -type f -print0 | xargs -0 grep -n 'Windows\.h'
./Modules/signalmodule.c:9:#include <Windows.h>
./Doc/using/windows.rst:341:   `A Python for Windows Tutorial <http://www.imladris.com/Scripts/PythonForWindows.html>`_
msg156608 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-03-22 19:27
re  Stefan Krah's posting:

I am sorry if I've caused any confusion.  I am building a Python extension (APSW), and it includes the sqlite "amalgamation" and also includes Python.h.

Sqlite includes Windows.h.  Which then causes the problem when Python.h is included.
msg156610 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-03-22 19:44
Jeff Robbins <report@bugs.python.org> wrote:
> I am sorry if I've caused any confusion.  I am building a Python extension
> (APSW), and it includes the sqlite "amalgamation" and also includes Python.h.

The Python build itself failed here, too, specifically in unicodeobject.c.

But I now see that apart from Windows.h there's also windows.h. Case
insensitivity is a difficult concept for a Unix user...
msg156613 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-03-22 20:14
To prevent further confusion. :)

db154e62ac03 -> OK
5fe7d19ec49a -> failure

5fe7d19ec49a with #undef patch -> OK


So I don't know if it is better to revert to db154e62ac03 or
to use the #undef patch.


Jeff, you originally said:

'An extension which includes both Windows.h and Python.h won't compile   due to the Windows.h definition of "small".'


Did you include Python.h before or after Windows.h? Extensions
are supposed to include Python.h before anything else.
msg156616 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-03-22 20:29
Stefan asked

'Did you include Python.h before or after Windows.h? Extensions
are supposed to include Python.h before anything else. '

I downloaded http://code.google.com/p/apsw/ to test it out on the latest sqlite release.  I followed the build instructions on its website and discovered this problem.  

While the code isn't mine, inspecting apsw.c, the main source file, I can see that the sqlite "amalgamation" is included BEFORE Python.h.  It is this "amalgamation" that, in turn, includes Windows.h.

Should I contact the extension's author(s)/maintainer(s) and tell them about this ordering requirement?
msg156624 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-03-22 22:57
Now unicodedata.c does not compile with VS2010, because somewhere, "small" is defined.
Do you mind if I change the _PyAccu.small member to 'smalls'?
msg156625 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-22 23:00
> Now unicodedata.c does not compile with VS2010, because somewhere, "small" is defined.
> Do you mind if I change the _PyAccu.small member to 'smalls'?

How about the #undef suggested somewhere?
msg156626 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-03-22 23:05
Sure, that's simpler.  I'll submit a fix.
msg156628 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-22 23:16
New changeset aff7ff2aae8c by Kristján Valur Jónsson in branch '3.2':
Issue #14387 : undefine 'small' so that it doesn't clash with Windows headers.
http://hg.python.org/cpython/rev/aff7ff2aae8c

New changeset 780aaa7b4b62 by Kristján Valur Jónsson in branch 'default':
Merge with 3.2 (Issue #14387)
http://hg.python.org/cpython/rev/780aaa7b4b62
msg157277 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-04-01 11:48
> Should I contact the extension's author(s)/maintainer(s) and tell them about this ordering requirement?

FWIW, it is the recommended way in the docs.



The Python build itself has been fixed. Does the http://code.google.com/p/apsw/ module build now?
msg157351 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-04-02 12:16
I'm happy to try another build of apsw, but am not up-to-speed with how to 
get development copies of Python.  As a user, I typically download the 
latest Windows .msi file and install it. If  you can point me to a document 
on how to proceed, I'll give it a try.

I'm Bcc'ing the APSW author in the event he can give it a try.

----- Original Message ----- 
From: "Stefan Krah" <report@bugs.python.org>
To: <jeffr@livedata.com>
Sent: Sunday, April 01, 2012 7:48 AM
Subject: [issue14387] Include\accu.h incompatible with Windows.h

Stefan Krah <stefan-usenet@bytereef.org> added the comment:

> Should I contact the extension's author(s)/maintainer(s) and tell them 
> about this ordering requirement?

FWIW, it is the recommended way in the docs.

The Python build itself has been fixed. Does the 
http://code.google.com/p/apsw/ module build now?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue14387>
_______________________________________
msg157356 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-04-02 13:31
Jeff Robbins <report@bugs.python.org> wrote:
> I'm happy to try another build of apsw, but am not up-to-speed with how to 
> get development copies of Python.  As a user, I typically download the 
> latest Windows .msi file and install it. If  you can point me to a document 
> on how to proceed, I'll give it a try.

http://www.python.org/ftp/python/3.3.0/python-3.3a2.msi should have the fix.
msg157370 - (view) Author: Jeff Robbins (jeffr@livedata.com) Date: 2012-04-02 17:23
Thanks for the pointer to the .msi.

The fix solved my apsw build problem.

I installed Python 3.3.a2 and apsw-3.7.11-r1.  I ran this build:

C:\Temp\apsw-3.7.11-r1>c:\Python33\python setup.py fetch --all 
build --enable-all-extensions install

apsw built and installed fine.  And my test app runs.

This line in accu.h did the trick:

#undef small /* defined by some Windows headers */

One of the apsw tests failed, but I don't think it had to do with the accu.h 
build problem.  Could be something I did wrong, or some issue between APSW 
and Python 3.3?

- Jeff

======================================================================
FAIL: testShell (tests.APSW)
Check Shell functionality
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Temp\apsw-3.7.11-r1\tests.py", line 6703, in testShell
    self.assertEqual(data, newdata)
AssertionError: Lists differ: [(3.1, 'xabc'), (3.2, 'xabfff"... != [(3.1, 
''), (
3.2, '')]

First differing element 0:
(3.1, 'xabc')
(3.1, '')

- [(3.1, 'xabc'), (3.2, 'xabfff"ffffc')]
+ [(3.1, ''), (3.2, '')]

----------------------------------------------------------------------
Ran 78 tests in 787.594s

FAILED (failures=1)

----- Original Message ----- 
From: "Stefan Krah" <report@bugs.python.org>
To: <jeffr@livedata.com>
Sent: Monday, April 02, 2012 09:31
Subject: [issue14387] Include\accu.h incompatible with Windows.h

Stefan Krah <stefan-usenet@bytereef.org> added the comment:

Jeff Robbins <report@bugs.python.org> wrote:
> I'm happy to try another build of apsw, but am not up-to-speed with how to
> get development copies of Python.  As a user, I typically download the
> latest Windows .msi file and install it. If  you can point me to a 
> document
> on how to proceed, I'll give it a try.

http://www.python.org/ftp/python/3.3.0/python-3.3a2.msi should have the fix.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue14387>
_______________________________________
msg157372 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-04-02 18:16
> The fix solved my apsw build problem.

Thanks for testing. As for the test failure, that could be really
anything. Often these kinds of failures turn out to be overly strict
assumptions in the test suite. Best ask the maintainer of the
package.



So 3.3 is covered. The fix isn't yet in 3.2.3rc2, but I guess
aff7ff2aae8c will go automatically into the final release.
msg157373 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-02 18:28
No it won't: but it's harmless enough that I think it can go into the final without creating another rc.
msg157374 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-02 18:29
Transplanted to f91ecbc8bafc in 3.2.3 release clone.
msg158018 - (view) Author: Roundup Robot (python-dev) Date: 2012-04-11 10:46
New changeset f91ecbc8bafc by Kristján Valur Jónsson in branch '3.2':
Issue #14387 : undefine 'small' so that it doesn't clash with Windows headers.
http://hg.python.org/cpython/rev/f91ecbc8bafc
msg159510 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-04-28 08:14
I think the issue is fixed in all affected branches. Georg, can we close it?
msg159511 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-28 08:22
I think so, yes.
History
Date User Action Args
2012-04-28 08:22:58georg.brandlsetstatus: pending -> closed

messages: + msg159511
2012-04-28 08:14:22skrahsetstatus: open -> pending
resolution: fixed
messages: + msg159510

stage: resolved
2012-04-11 10:46:24python-devsetmessages: + msg158018
2012-04-02 18:29:24georg.brandlsetmessages: + msg157374
2012-04-02 18:28:42georg.brandlsetmessages: + msg157373
2012-04-02 18:16:05skrahsetmessages: + msg157372
2012-04-02 17:23:49jeffr@livedata.comsetmessages: + msg157370
2012-04-02 13:31:20skrahsetmessages: + msg157356
2012-04-02 12:16:58jeffr@livedata.comsetmessages: + msg157351
2012-04-01 11:48:08skrahsetmessages: + msg157277
2012-03-22 23:16:43python-devsetmessages: + msg156628
2012-03-22 23:05:15kristjan.jonssonsetmessages: + msg156626
2012-03-22 23:00:53pitrousetmessages: + msg156625
2012-03-22 22:57:30kristjan.jonssonsetnosy: + kristjan.jonsson
messages: + msg156624
2012-03-22 20:29:21jeffr@livedata.comsetmessages: + msg156616
2012-03-22 20:14:56skrahsetmessages: + msg156613
2012-03-22 19:44:21skrahsetmessages: + msg156610
2012-03-22 19:27:29jeffr@livedata.comsetmessages: + msg156608
2012-03-22 19:22:45skrahsetfiles: + issue14387.diff
keywords: + patch
messages: + msg156607
2012-03-22 17:50:30jeffr@livedata.comsetmessages: + msg156598
2012-03-22 17:21:38pitrousetassignee: pitrou ->
messages: + msg156596
2012-03-22 16:24:00georg.brandlsetmessages: + msg156590
2012-03-22 16:22:38skrahsetnosy: + skrah
messages: + msg156589
2012-03-22 16:20:34georg.brandlsetmessages: + msg156588
2012-03-22 13:49:31pitrousetmessages: + msg156561
2012-03-22 13:48:54python-devsetnosy: + python-dev
messages: + msg156560
2012-03-22 12:41:44pitrousetpriority: normal -> release blocker
nosy: + georg.brandl, benjamin.peterson

versions: + Python 3.3
2012-03-22 12:19:13amaury.forgeotdarcsetassignee: pitrou

messages: + msg156547
nosy: + amaury.forgeotdarc, pitrou
2012-03-22 11:53:03jeffr@livedata.comcreate