I'm attempting to package Python 3 for a Linux distribution, together with a stack of python extension 
modules; I'm currently using Python-3.1.1.   (see https://fedoraproject.org/wiki/Features/Python3F13 )

Many of these extension modules are close to compiling under both python 2 and python 3.

The page "http://wiki.python.org/moin/PortingExtensionModulesToPy3k" refers to using the macros in 
intobject.h, so that all PyInt_* calls in the python 3 build are aliased to PyLong_ API hooks.

Similarly, the page: http://docs.python.org/howto/cporting.html recommends using this file.

However, that header file was removed in this commit:
  http://svn.python.org/view?view=rev&revision=71697
with this message:
  Issue #4910:  PyNumber_Int is deprecated in 3.0.1; will be removed in 3.1.
(which links to this issue: http://bugs.python.org/issue4910 ; that issue refers to removal of the 
nb_long slot).

This seems to make it harder to port modules.

Is is acceptable if I ship that header file in my distribution packages of python-3.1.1 ?  (possibly 
with a reworded deprecation warning?)

Alternatively, is the fix to migrate all usage of the PyInt_ API to the PyLong_ equivalents ?  That 
would cause a change of behavior for the python 2 builds, assuming a shared source tree.

(I'd much prefer to ship the latest in the py3k branch than to stay with 3.0.1 for this)


In any case, it seems like the porting documentation isn't in sync with the code.

Hope this is helpful.

> However, that header file was removed in this commit:
>  http://svn.python.org/view?view=rev&revision=71697

Hmm.  That was me.

> with this message:
>   Issue #4910:  PyNumber_Int is deprecated in 3.0.1; will be removed in
> 3.1.

That's the message for r69517, I think, which just added an entry to
Include/intobject.h.  The message for r71697 was:

"The comments at the top of intobject.h say that it will be removed in
3.1.  Make it so."

I'm not too sure of the history here, but I suspect that
Include/intobject.h was at least partly an aid to porting the Python
extension modules from 2.x to 3.0, rather than third-party modules.

Christian, Benjamin:  any comments?

I'm -1 on resurrecting Include/intobject.h in its old location.  I don't
much like the idea of having a random include file that's no longer used
by Python itself in the main Include directory;  it'll likely succumb to
bitrot before long.  But maybe there's a place for 2-to-3 porting aids
in the Tools directory, where there's less expectation that files are
well-maintained?

> Is is acceptable if I ship that header file in my distribution
> packages of python-3.1.1 ?  (possibly with a reworded deprecation
> warning?)

I can't see any problem with this.

I agree the porting documentation needs to be updated, whatever happens.

I didn't mean to assign this to Georg.  Apologies.

You didn't.  Doc bugs are automatically assigned to Georg by the tracker.

Hmm, I wish intobject.h hadn't been removed so soon. I'm not really sure
how a file of #defines could suffer bitrot. This point is probably moot,
though because there's little point in having its presence skip a
version. I suppose sticking it in Tools or even Doc/includes is the
second best option.

[Benjamin]
>I wish intobject.h hadn't been removed so soon.

Yes;  I'm sorry about that.

> I'm not really sure how a file of #defines could suffer bitrot.

Good point.  Me neither.

I don't think it would hurt to put it back, would it?  I think the "remove 
this in 3.1" note was made when we expected 3.1 to be happening 1.5 years 
after 3.0 rather than a few months.

MvL made this comment in
http://www.mail-archive.com/python-dev@python.org/msg43844.html

I'm copying it here so it doesn't get lost and because I think he makes
a good point that many people would miss (at least I didn't think of it).
-----------------------------------------------

The macros (unfortunately) allowed
to make non-obvious mistakes. Now that they are gone, people need to
really think of what precisely they want to do.

For example, consider

if (PyInt_Check(o)){
  long val = PyInt_AsLong(o);
  // process
} else if (PyLong_Check(o)) {
  long long val = PyLong_AsLongLong(o);
  // check for overflow
  // process
}

With intobject.h, this code would continue to compile, but work
incorrectly, as the second case will never be executed. It would
be better to port this as

#if Py2.x
if (PyInt_Check(o)){
  long val = PyInt_AsLong(o);
  // process
} else
#endif
if (PyLong_Check(o)) {

i.e. eliminating the int case altogether. For another example,

long foo = PyInt_AsLong(Foo);

has a hidden error in 3.x, with intobject: PyLong_AsLong might
overflow, which the 2.x case doesn't.

So eliminating intobject.h likely helps avoiding subtle errors.

Regards,
Martin

Eric pointed me to this ticket after having raised the question on
python-dev: http://www.mail-archive.com/python-dev@python.org/msg43841.html

I think the discussion should be continued there instead of on this ticket.

Just for completeness, I'm copying my reply to Martin's reply here
(http://www.mail-archive.com/python-dev@python.org/msg43849.html):

"""
> For example, consider
> 
> if (PyInt_Check(o)){
>   long val = PyInt_AsLong(o);
>   // process
> } else if (PyLong_Check(o)) {
>   long long val = PyLong_AsLongLong(o);
>   // check for overflow
>   // process
> }
> 
> With intobject.h, this code would continue to compile, but work
> incorrectly, as the second case will never be executed. It would
> be better to port this as
> 
> #if Py2.x
> if (PyInt_Check(o)){
>   long val = PyInt_AsLong(o);
>   // process
> } else
> #endif
> if (PyLong_Check(o)) {
> 
> i.e. eliminating the int case altogether.

Sure, but that assumes that the original code already had support
for Python longs, which a lot of code doesn't.

In an ideal world, developers would add that code to their
extensions right away. In the real world, where developers only
have limited resources available, you'll get more 3.x ports
by making such ports as painless as possible while at the
same time not forcing them to alienate their 2.x user base.

The long support could then be added in later releases
of the extensions, giving the developers more time adapt.

> For another example,
> 
> long foo = PyInt_AsLong(Foo);
> 
> has a hidden error in 3.x, with intobject: PyLong_AsLong might
> overflow, which the 2.x case doesn't.

That's not quite true: PyInt_AsLong(obj) will try the
nb_int slot on non-integer objects which can return errors
(it returns -1 and sets the error message).

> So eliminating intobject.h likely helps avoiding subtle errors.

In the long run, yes. In the short run, other criteria are
more important, IMHO.
"""

IMO, it would be worthwhile collecting all Python 2.x compatibility C
APIs in two new files:

 * py2compat.h
 * py2compat.c

These could then be used in extensions and make the use of such
compatibility APIs explicit in the extension.

Hi,

Might I suggest that, whatever the outcome of the re-adding intobject.h
discussion, the documentation is updated? I think I'm not the only
module author which spent time trying to understand why the 3.1
documentation refers to non-existent header before finally finding this
bug :)

thanks a lot,
iustin

Bump... is this still valid?

I tend to agree with the argument that the removal of intobject.h was
a good thing, since it avoids subtle errors.

Probably no one wants to reinstate intobject.h at this point, so unless
there are objections, I'll update the docs in a couple of days.

Go ahead!

The intobject header file used to make porting to Python 3 easier. Nowadays it's no longer required.

New changeset 6df456f8fc6d by Stefan Krah in branch '3.3':
Issue #7353: Remove references to Include/intobject.h in the C-porting howto.
http://hg.python.org/cpython/rev/6df456f8fc6d