I think it'd be useful with a built-in module (I
propose the name 'mint' :), providing classes for
emulating machine ints. There should be support for
ints of different sizes (at least 8, 16, 32, 64),
signedness and endianness. The machine ints would react
to overflow (negative values) and bit-level operations
in the same way their C equivalents would.

One idea for a feature would be a intobj.pack() (or
str() or whatever) method that could be used instead of
the struct module's pack(). int16(12345).pack() ->
"\x30\x39". int32(-1).pack() -> "\0xff\0xff\0xff\0xff",
etc.

A related idea is that these ints could provide slicing
to extract individual sets of bits or bytes. For
example, uint32()[8:16] could return an int (uint8?)
made from the bits in the int's second byte (whether
it's the second from the left or the right might be
fuzzy, I do realize).

Applications for this would be algorithmic code
involving bit fiddling (useful for cryptographic
applications, perhaps) and, obviously, programs that
have to operate on any binary data.  It might also
resolve some issues related to the unification of longs
and ints.

Logged In: YES 
user_id=80475

FTI, there was a related feature request,
www.python.org/sf/846568 for treating numbers as bitarrays.
 As proposed, it was invalid because numbers in Python are
immutable (no bit assignments allowed).

That could be combined with this idea generic numeric
register class that optionally specifies size (if bound),
base, endianess, etc.  

bcdreg = register(places=5, base=10)
bcdreg.assign(697)
bcdreg[-1] --> 7
bcdreg.multiply(1000) --> 97000

None of this is for the mainstream user and it would be
unlikely to ever be included in the standard library.  The
best approach would be to experiment with an API in pure
python, perhaps publish it as a recipe, and if it is well
received, put a C version on the vaults of parnassus.

Logged In: YES 
user_id=756835

As for mainstream uses, I think replacing struct.pack would
be one.

Logged In: YES 
user_id=11105

You might look at the ctypes module (google for it) - it
implements mutable C compatible data types.

Logged In: YES 
user_id=139309

If I were providing slicing on bits I would say that foo[n] is bool(foo & 
(1<<n)) .. which means bit 0 is least significant, and bit N is most 
significant.  I don't think any other way would make sense.

Logged In: YES 
user_id=4771

A number of projects end up redefining Python classes that
try to emulate the raw integers of C, which is a pain to do
properly in Python. I think that a general-purpose 'mint'
class would be generally useful.

Writing them in Python is a bit of a pain
(http://codespeak.net/svn/pypy/trunk/src/pypy/objspace/std/restricted_int.py).
Pyrex seems to be a good way to generate it as a extension
module
(http://codespeak.net/svn/user/arigo/misc/machineint.pyx).

Is it be feasible to add arithmetic operations
to the ctypes integer types?  Since ctypes is now
in the core, it would seem better to enhance ctypes
than provide a new module.

I think this would be valuable for rapid prototyping
of an algorithm in Python before translating it into
C.  In particular, it might help with detecting some
common C code bugs involving undefined behaviour---for
example, overflow in signed-integer arithmetic.

I wonder if a patch for ctypes like this (which is not yet complete)
could be used to implement this, or MUST it be implemented in C?

The patch contains a mixin class that implements the numeric methods. 
However, the actual operation takes place in Python; only the operands
are converted into ctypes types first, the operand is applied to the
value, and the result is converted to a ctypes instance again.

One difficulty with the patch is that the original ctypes code contained
a tp_as_number member where only the nb_nonzero slot was actually
implemented; this prevented the mixin class to do it's work (Python
didn't call the special methods.  I wonder if there is a bug somewhere...).

> I wonder if a patch for ctypes like this (which is not yet complete)
> could be used to implement this, or MUST it be implemented in C?

For the intended purpose of giving access to raw machine arithmetics, I 
would think a C implementation would be required, but Theller's patch is 
a good start.

> One difficulty with the patch is that the original ctypes code
> contained a tp_as_number ...

This can be solved by changing the order of bases for c_<type> classes.  
See attached.

The next step would be to rewrite _NumberMixin in C.  C code can be 
limited to methods operating on known c_ types, say ll_add(x, y) to add 
two longs and hl_cast(x) to cast short to long with python code taking 
care of the dispatch.  (What's the latest on the multiple dispatch, BTW?
)

>> One difficulty with the patch is that the original ctypes code
>> contained a tp_as_number ...

>This can be solved by changing the order of bases for c_<type> classes.  
>See attached.

Cool! Why didn't I think of that myself?

Attaching a patch that implements machine arithmetics for numeric ctypes in C.

I don't care much for ctypes but I think the amount of code duplication in that patch is scary.

Code duplication is unavoidable because the goal is to give access to
machine arithmetics which means (# types) x (# operations) of very
similar looking functions. I considered reducing (perceived) code
duplication through some pre-processor magic, but doing so would make
code much harder to understand and almost impossible to debug.

On Thu, Mar 4, 2010 at 7:06 PM, Antoine Pitrou <report@bugs.python.org> wrote:
>
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> I don't care much for ctypes but I think the amount of code duplication in that patch is scary.
>
> ----------
> nosy: +pitrou
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue887237>
> _______________________________________
>

Le Fri, 05 Mar 2010 02:21:30 +0000,
Alexander Belopolsky <report@bugs.python.org> a écrit :
> 
> Code duplication is unavoidable because the goal is to give access to
> machine arithmetics which means (# types) x (# operations) of very
> similar looking functions. I considered reducing (perceived) code
> duplication through some pre-processor magic, but doing so would make
> code much harder to understand and almost impossible to debug.

Carefully written macros shouldn't be hard to understand.
At least there wouldn't be the risk of overlooking one of the methods
when making modifications.

Look at Modules/operator.c for example of an effective use of macros to minimize code duplication.

I would like to hear from Thomas before introducing macros in this
code.  I tried to follow the style of cfield.c which shows similar
code duplication.

There are also some questions that need to be answered before
polishing the code.

1. Should mixed arithmetics be supported?
2. Should we do anything special about floating point operations?
Wrapping them in PyFPE_START/STOP_PROTECT?
3. Should we support in-place operations?
4. Bitwise operations on integers?

On Thu, Mar 4, 2010 at 9:29 PM, Raymond Hettinger
<report@bugs.python.org> wrote:
>
> Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment:
>
> Look at Modules/operator.c for example of an effective use of macros to minimize code duplication.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue887237>
> _______________________________________
>

>> Code duplication is unavoidable because the goal is to give access to
>> machine arithmetics which means (# types) x (# operations) of very
>> similar looking functions. I considered reducing (perceived) code
>> duplication through some pre-processor magic, but doing so would make
>> code much harder to understand and almost impossible to debug.
> 
> Carefully written macros shouldn't be hard to understand.
> At least there wouldn't be the risk of overlooking one of the methods
> when making modifications.

The code duplication in Alex's patch is indeed enormous.  I can see
two solutions to avoid (or work around) them.  Use of macros, or writing
a Python script that generates the C-code.  Anyway, I guess his patch
is some example code, not more.  (Using C++ overloaded functions or even
templates would be another - but only theoretical - possibility).

> I would like to hear from Thomas before introducing macros in this
> code.  I tried to follow the style of cfield.c which shows similar
> code duplication.
> 
> There are also some questions that need to be answered before
> polishing the code.
> 
> 1. Should mixed arithmetics be supported?
> 2. Should we do anything special about floating point operations?
> Wrapping them in PyFPE_START/STOP_PROTECT?
> 3. Should we support in-place operations?
> 4. Bitwise operations on integers?

I would answer 1, 3, and 4, with 'yes'. However, this grows the code size
by another dimension since we now have #types * #operation * #types.

My answer for question 3 is 'I don't know'.

Before I forget:  It may be possible to implement ctypes number methods
as a third-party module by implementing a mixin class, and replacing the
c_... type definitions in Lib/ctypes/__init__.py.

Attaching a patch that is equivalent toissue887237.diff, but uses preprocessor to generate repetitive code.

Perhaps people could review the latest patch in view of the following.

1) Quote from Mark Dickinson msg75815

"I think this would be valuable for rapid prototyping
of an algorithm in Python before translating it into
C.  In particular, it might help with detecting some
common C code bugs involving undefined behaviour---for
example, overflow in signed-integer arithmetic."

2) issue1621 "Do not assume signed integer overflow behavior"

Perhaps this could be treated as purely experimental?  (This might have been suggested already, sorry if I've missed it). Whatever happens I'd be glad to help out if possible, the big asset I have compared to others is time as I only work part time.

As far as I understand, the main concern about issue887237.diff was code duplication.  There are two ways to fight it: C preprocessor tricks as in issue887237-macro.diff and code generation as done in numpy.  With improved macro support in many compilers and debuggers, preprocessor approach may be preferable.  However I cannot move this issue forward until those who objected to code duplication review the latest patch.  Unassigning.

You could write a Python script to generator the methods.