I propose the addition of three new macros in pyport.h

#define PY_LITTLE_ENDIAN 1234
#define PY_BIG_ENDIAN 4321
#define PY_BYTE_ORDER <either little or big endian>

that are either set by a configure test or implemented like brg_endian.h. pyconfig.h has WORDS_BIGENDIAN which just checks for __BIG_ENDIAN__. The sysmodule has its own C code that checks the first byte of a long to detect the endianess.

http://hg.python.org/cpython/file/dd5e98ddcd39/Modules/_sha3/keccak/brg_endian.h

I'd like to remove Keccak's brg_endian.h (#16113 SHA-3 module) for something more official and useful to other code. I had to re-add the file because the tests were failing on a big endian Solaris (Sparc) machine.

I agree, it is worth bring order to work with endianess. We can get rod of BYTEORDER_IS_BIG_ENDIAN and BYTEORDER_IS_LITTLE_ENDIAN in Objects/unicodeobject.c, IS_LITTLE_ENDIAN in Objects/longobject.c, tests in other modules. I don't see the necessity in new macros. WORDS_BIGENDIAN is enough.

If something failing on a big endian Solaris, build script should just define WORDS_BIGENDIAN.

Here is a patch which removes some unnecessary defines and tests.

This doesn't depend on #16113 (quite the contrary).

> #define PY_LITTLE_ENDIAN 1234
> #define PY_BIG_ENDIAN 4321
> #define PY_BYTE_ORDER <either little or big endian>

Ouch, sounds confusing. I would rather have PY_LITTLE_ENDIAN defined only on little-endian machines and PY_BIG_ENDIAN only on big-endian machines.
(and PY_BYTE_ORDER isn't necessary)

> Ouch, sounds confusing. I would rather have PY_LITTLE_ENDIAN defined only
> on little-endian machines and PY_BIG_ENDIAN only on big-endian machines.
> (and PY_BYTE_ORDER isn't necessary)

Why use two complementary boolean variables for a single boolean value (Python 
does not support mixed endian in any case)? There is WORDS_BIGENDIAN already.

> > Ouch, sounds confusing. I would rather have PY_LITTLE_ENDIAN defined only
> > on little-endian machines and PY_BIG_ENDIAN only on big-endian machines.
> > (and PY_BYTE_ORDER isn't necessary)
> 
> Why use two complementary boolean variables for a single boolean value (Python 
> does not support mixed endian in any case)?

Because it's simply more practical than having to remember which one
exists :-)

> There is WORDS_BIGENDIAN already.

A rather unfortunate IMHO.

Just a reminder that we support configurations where there are both big-endian and little-endinan binaries in the *same* executable file: for example, Mac OS X 32-bit-only universal binaries (.so, .dylib, .exe) contain both i386 and ppc code in the same file.  So any -endian differences must remain isolated to C code or be dynamically tested at execution time in Python code. And ./configure time tests can be problematic.  (The proposed patch doesn't have that problem but it might be an issue later in tests.)

My proposal mimics the API of endian.h. It defines a BYTE_ORDER macro that is either equal to LITTLE_ENDIAN, BIG_ENDIAN or PDB_ENDIAN (aka mixed endian). I need it that way for Keccak but I can roll my own definitions if you prefer just two macros.

If you prefer two macros instead of three then we should name them PY_IS_LITTLE_ENDIAN and PY_IS_BIG_ENDIAN.

Ned:
If I understand you correctly than we can't have a configure definition and need macro magic like brg_endian.h.

Christian: That's right because there is only one configure execution in the OS X universal builds and only compiler call per module just like a normal single-architecture unix build.  Under the covers, the Apple compiler driver transparently makes multiple compiler calls, one for each architecture, and then use lipo to combine the binaries into one .o file etc.

> If you prefer two macros instead of three then we should name them
> PY_IS_LITTLE_ENDIAN and PY_IS_BIG_ENDIAN.

Fine with me. Having three macros is pointless complication.

> If I understand you correctly than we can't have a configure definition 
> and need macro magic like brg_endian.h.

Replicating the current WORDS_BIGENDIAN logic should be enough.

I think it's not enough as some platforms prefix/suffix the macros with dashes. The values must be compared to BYTE_ORDER iff the macro is defined. too. I've checked some machines of the Snakebite network and came up with this macro cascade:

/*
 * The endianess (byte order) can be defined in several ways. Some platforms
 * define either LITTLE_ENDIAN or BIG_ENDIAN while other platforms define
 * both and require comparison to BYTE_ORDER. Sometimes the macros are
 * prefixed or suffixed with one or two dashes, too. SPAM seems to be an
 * alias of __SPAM an all platforms that have SPAM.
 * The endian check can't be handled in a configure step as it would break
 * Apple's universal builds.
 */

#if defined(__BYTE_ORDER)
#  if defined(__BIG_ENDIAN) && __BYTE_ORDER == __BIG_ENDIAN
#    define PY_IS_BIG_ENDIAN 1
#  elif defined(__LITTLE_ENDIAN) && __BYTE_ORDER == __LITTLE_ENDIAN
#    define PY_IS_LITTLE_ENDIAN 1
#  else
#    error "__BYTE_ORDER not in (__BIG_ENDIAN, __LITTLE_ENDIAN)"
#  endif
#elif defined(__BIG_ENDIAN) && !defined(__LITTLE_ENDIAN)
#  define PY_IS_BIG_ENDIAN 1
#elif defined(__LITTLE_ENDIAN) && !defined(__BIG_ENDIAN)
#  define PY_IS_LITTLE_ENDIAN 1
/* and know the same block with just one dash as prefix */
#elif defined(_BYTE_ORDER)
#  if defined(_BIG_ENDIAN) && _BYTE_ORDER == _BIG_ENDIAN
#    define PY_IS_BIG_ENDIAN 1
#  elif defined(_LITTLE_ENDIAN) && _BYTE_ORDER == _LITTLE_ENDIAN
#    define PY_IS_LITTLE_ENDIAN 1
#  else
#    error "_BYTE_ORDER not in (_BIG_ENDIAN, _LITTLE_ENDIAN)"
#  endif
#elif defined(_BIG_ENDIAN) && !defined(_LITTLE_ENDIAN)
#  define PY_IS_BIG_ENDIAN 1
#elif defined(_LITTLE_ENDIAN) && !defined(_BIG_ENDIAN)
#  define PY_IS_LITTLE_ENDIAN 1
/* extra apple */
#elif defined(__APPLE__)
#  if defined(__BIG_ENDIAN__)
#    define PY_IS_BIG_ENDIAN 1
#  elif defined(__LITTLE_ENDIAN__)
#    define PY_IS_LITTLE_ENDIAN 1
#  else
#    error "__APPLE__ but neither __BIG_ENDIAN__ nor __LITTLE_ENDIAN__"
#  endif
/* Windows */
#elif defined(WIN32)
#  define PY_IS_LITTLE_ENDIAN 1
#else
#  error "Unable to detect endianess"
#endif

> Fine with me. Having three macros is pointless complication.

Don't having two macros for one boolean value pointless complication too?

> > Fine with me. Having three macros is pointless complication.
> 
> Don't having two macros for one boolean value pointless complication too?

No, since you need only one to get going. Either:

#ifdef PY_LITTLE_ENDIAN
...

or:

#ifdef PY_BIG_ENDIAN
...

So this is really a simplification compared to the awkward:

#ifdef PY_ENDIAN == BIG_ENDIAN

Usually you need code for both alternatives.

#ifdef WORDS_BIGENDIAN
...
#else
...
#endif

What can be simpler? Of cause, using *two* macros is complicated. You need 
only one and second macros is unnecessary.

AC_C_BIGENDIAN

As Victor point configure script already check for endian (macro AC_C_BIGENDIAN) but you should ask python OS X guru to review . 
Macro is not adjusted to python needs as action for universal build is not defined yet. The default is to define AC_APPLE_UNIVERSAL_BUILD .

Otherwise WORDS_BIGENDIAN is defined for bigendian .

The patch defines either PY_IS_BIG_ENDIAN or PY_IS_LITTLE_ENDIAN and sets PY_LITTLE_ENDIAN_FLAG to either 0 or 1 for some modules that require a boolean flag.

I would prefer if you yanked PY_LITTLE_ENDIAN_FLAG. We don't need two different ways to do it.

I've replaced the macros with PY_BIG_ENDIAN or PY_LITTLE_ENDIAN. Both are always defined and exactly one is set to 1, the other to 0.

Looks fine to me. I suppose you'll have to check the big endian buildbots.

New changeset d2127cdec10e by Christian Heimes in branch 'default':
Issue #16166: Add PY_LITTLE_ENDIAN and PY_BIG_ENDIAN macros and unified
http://hg.python.org/cpython/rev/d2127cdec10e

> I've replaced the macros with PY_BIG_ENDIAN or PY_LITTLE_ENDIAN. Both are
> always defined and exactly one is set to 1, the other to 0.

It looks like a joke.

The programmer puts on the table by his bed two glasses. One with water, for 
the case of he will drink at night. Another empty, for the case of he will not 
drink.

Instead of using macro values, I would rather define ONLY a single macro, PY_LITTLE_ENDIAN or PY_BIG_ENDIAN.

Far less error prone!.

It is far fool-proof "#ifdef PY_LITTLE_ENDIAN" than "#if PY_LITTLE_ENDIAN".

> Instead of using macro values, I would rather define ONLY a single macro,
> PY_LITTLE_ENDIAN or PY_BIG_ENDIAN.

Time machine strikes back.

#ifdef WORDS_BIGENDIAN