Michael Foord reminded me of this issue recently. It was discussed on pydev a few years back and met with limited enthusiasm. I speak from experience in live production with EVE that this simple change saved us a lot of memory. Integral floating point numbers are surprisingly common, falling out of mathematical operations and when reading tabular data. 0.0 and 1.0 in particular.

Does it not decrease the performance?

Falling out integral floating point numbers in the mathematical floating point calculations seems unlikely. I suggest interning integral floats only in conversions str -> float and int -> float. Exception can be made to zero, which can result in operations with zero, and which simple tested.

This looks like a duplicate of bpo-4024.

Yes, there is a measurable performance decrease in pybench arithmetic tests.

Integers don't fall out of arithmetic that often, true. But integral floats are incredibly common in tabular data. In a game such as Eve Online, configuration data contains a lot of 0.0, 1.0, -1.0 and so on. This patch saved us many megabytes on the server. I can check again...

>>> [sys.getrefcount(float(i)) for i in range(-10, 11)]
[777, 4, 38, 9, 215, 691, 627, 185, 98, 603, 73180, 62111, 8326, 6225, 6357, 11737, 2906, 1393, 3142, 1145, 5601]
>>> sum([sys.getrefcount(float(i)) for i in range(-10, 11)])
185340
>>> 
This is on an idle server.  A server with lots of stuff going on will have this:
[16715, 184, 1477, 34, 1505, 27102, 3878, 1344, 6248, 974, 595889, 313062, 124072, 120054, 65585, 138667, 13265, 2499, 15677, 3175, 24821]
>>> sum([sys.getrefcount(float(i)) for i in range(-10, 11)])
1465155

About half of the interned floats are 0.0
On a 64 bit machine with each float taking 24 bytes, this is 35mb net.

An alternative could be to add a function for manual intering of floating point data, which one can use e.g. when reading tabular data.

Integers don't fall out of arithmetic that often, true. But integral
floats are incredibly common in tabular data. In a game such as Eve
Online, configuration data contains a lot of 0.0, 1.0, -1.0 and so
on. This patch saved us many megabytes on the server.

Can't you do your own interning when reading configuration data? You can even do it in pure Python.

If CPython starts interning some integral floats, I think only {-1.0, -0.0, 0.0, 1.0} should apply. The rest is too application-specific.

The -10..11 range was determined empirically. As you see from the values, only -10 shows up as significant... In fact, just storing 0 to 5 would capture the bulk of the savings.

Right, I'll do some test with the hardcoded values you mentioned.
Btw, I don't think -0.0 is worth it, that value is a typical arithmetic result and not something you typically get from input data.

Also, while interning in python is certainly possible, perhaps it would make sense to implement such in c. Perhaps in the math module, or perhaps even the 'intern' builtin can be extended for this purpose? It could be documented to intern any immutable object that it sees fit to intern, leaving the details to the implementation.

Right, I'll do some test with the hardcoded values you mentioned.
Btw, I don't think -0.0 is worth it, that value is a typical
arithmetic result and not something you typically get from input data.

I was suggesting -0.0 in the hope that it might make the patch a bit
simpler, but I was probably wrong: you still have to test the float's
sign to distinguish between the two values.

Also, while interning in python is certainly possible, perhaps it
would make sense to implement such in c.

I think it makes sense in the float implementation if it doesn't
significantly decrease performance. I suggest you post the benchmark
numbers (see http://hg.python.org/benchmarks/ ).

Adding a separate builtin/stdlib function would be overkill IMO.

Try moving your changes from PyFloat_FromDouble to PyFloat_FromString. Look at memory and perfomance.

PyFloat_FromString() is very restrictive. In our case, for example, the floats in question don't come from text files.

I'm adding a new file with changes suggested by Antoine. only 0.0 and 1.0 are interned. It turns out that the "int" test is expensive, and special casing it to check for '0.0' and '1.0' with direct float comparison makes the performance hit go away. Here are the figures:

D:\pydev\hg\cpython2>pcbuild10\python.exe Tools\pybench\pybench.py -s interned8 -c normal
-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------

• using CPython 3.3.0a1+ (default, Mar 22 2012, 15:39:50) [MSC v.1600 32 bit (Intel)]
• disabled garbage collection
• system check interval set to maximum: 2147483647
• using timer: time.clock

-------------------------------------------------------------------------------
Benchmark: interned8
-------------------------------------------------------------------------------

Rounds: 10
Warp:   10
Timer:  time.clock

Machine Details:
   Platform ID:    Windows-7-6.1.7601-SP1
   Processor:      Intel64 Family 6 Model 26 Stepping 5, GenuineIntel

Python:
   Implementation: CPython
   Executable:     D:\pydev\hg\cpython2\pcbuild10\python.exe
   Version:        3.3.0a1+
   Compiler:       MSC v.1600 32 bit (Intel)
   Bits:           32bit
   Build:          Mar 22 2012 15:39:50 (#default)
   Unicode:        UCS4

-------------------------------------------------------------------------------
Comparing with: normal
-------------------------------------------------------------------------------

Rounds: 10
Warp:   10
Timer:  time.clock

Machine Details:
   Platform ID:    Windows-7-6.1.7601-SP1
   Processor:      Intel64 Family 6 Model 26 Stepping 5, GenuineIntel

Python:
   Implementation: CPython
   Executable:     D:\pydev\hg\cpython2\pcbuild10\python.exe
   Version:        3.3.0a1+
   Compiler:       MSC v.1600 32 bit (Intel)
   Bits:           32bit
   Build:          Mar 22 2012 13:53:45 (#default)
   Unicode:        UCS4

Test minimum run-time average run-time
this other diff this other diff
-------------------------------------------------------------------------------
BuiltinFunctionCalls: 59ms 59ms -0.7% 60ms 60ms -0.3%
BuiltinMethodLookup: 40ms 40ms -0.2% 40ms 40ms -0.2%
CompareFloats: 47ms 46ms +2.3% 47ms 46ms +2.3%
CompareFloatsIntegers: 125ms 126ms -0.8% 129ms 131ms -2.0%
CompareIntegers: 76ms 76ms -0.2% 76ms 76ms -0.3%
CompareInternedStrings: 60ms 60ms -0.0% 61ms 61ms +0.2%
CompareLongs: 44ms 44ms -0.4% 44ms 44ms -0.4%
CompareStrings: 59ms 58ms +2.1% 59ms 58ms +1.8%
ComplexPythonFunctionCalls: 59ms 60ms -1.7% 60ms 61ms -2.5%
ConcatStrings: 49ms 55ms -10.7% 49ms 55ms -10.6%
CreateInstances: 67ms 66ms +1.1% 68ms 68ms -0.1%
CreateNewInstances: 50ms 50ms +1.3% 51ms 50ms +0.4%
CreateStringsWithConcat: 70ms 71ms -1.3% 70ms 72ms -1.7%
DictCreation: 45ms 46ms -1.7% 46ms 46ms -1.5%
DictWithFloatKeys: 81ms 84ms -2.8% 81ms 84ms -2.7%
DictWithIntegerKeys: 52ms 55ms -5.4% 52ms 55ms -5.1%
DictWithStringKeys: 47ms 51ms -6.6% 48ms 51ms -6.6%
ForLoops: 42ms 42ms +0.2% 42ms 42ms +0.2%
IfThenElse: 63ms 59ms +5.5% 63ms 59ms +5.6%
ListSlicing: 39ms 39ms +0.0% 39ms 39ms -0.4%
NestedForLoops: 67ms 67ms -0.2% 68ms 68ms +0.0%
NestedListComprehensions: 63ms 62ms +1.2% 64ms 63ms +1.2%
NormalClassAttribute: 102ms 100ms +1.3% 102ms 101ms +1.4%
NormalInstanceAttribute: 62ms 62ms +0.2% 63ms 63ms +0.2%
PythonFunctionCalls: 60ms 60ms +0.8% 61ms 60ms +1.3%
PythonMethodCalls: 74ms 74ms -0.0% 74ms 74ms -0.3%
Recursion: 103ms 102ms +1.4% 104ms 102ms +1.3%
SecondImport: 109ms 109ms -0.4% 110ms 110ms -0.6%
SecondPackageImport: 111ms 112ms -0.6% 113ms 113ms +0.2%
SecondSubmoduleImport: 154ms 153ms +0.7% 155ms 155ms -0.2%
SimpleComplexArithmetic: 56ms 54ms +3.7% 57ms 56ms +1.9%
SimpleDictManipulation: 94ms 93ms +0.3% 94ms 94ms -0.2%
SimpleFloatArithmetic: 45ms 44ms +3.2% 45ms 45ms +1.7%
SimpleIntFloatArithmetic: 57ms 56ms +1.8% 57ms 57ms +0.1%
SimpleIntegerArithmetic: 56ms 56ms +1.2% 57ms 56ms +1.0%
SimpleListComprehensions: 53ms 52ms +1.8% 54ms 53ms +2.4%
SimpleListManipulation: 49ms 53ms -8.0% 50ms 53ms -6.7%
SimpleLongArithmetic: 41ms 40ms +1.6% 41ms 41ms +0.1%
SmallLists: 66ms 68ms -2.8% 67ms 68ms -2.5%
SmallTuples: 76ms 72ms +5.5% 76ms 72ms +5.8%
SpecialClassAttribute: 144ms 140ms +2.8% 145ms 142ms +2.1%
SpecialInstanceAttribute: 64ms 64ms +0.7% 65ms 64ms +1.1%
StringMappings: 126ms 132ms -4.2% 127ms 132ms -4.0%
StringPredicates: 71ms 70ms +2.3% 72ms 70ms +2.5%
StringSlicing: 66ms 67ms -0.6% 67ms 67ms -1.1%
TryExcept: 44ms 43ms +1.5% 44ms 44ms +0.7%
TryFinally: 49ms 53ms -7.4% 49ms 54ms -8.1%
TryRaiseExcept: 24ms 22ms +8.6% 25ms 23ms +8.9%
TupleSlicing: 61ms 61ms -0.7% 61ms 62ms -0.5%
WithFinally: 65ms 65ms -0.3% 65ms 65ms -0.0%
WithRaiseExcept: 62ms 62ms -0.4% 62ms 63ms -0.6%
-------------------------------------------------------------------------------
Totals: 3448ms 3454ms -0.2% 3474ms 3487ms -0.4%

(this=interned8, other=normal)

D:\pydev\hg\cpython2>

Ok. In the current state it doesn't look detrimental, although I'm not convinced it's that useful either.
That said, the implementation needs a bit of work, I don't think it's ok for float_is_positive_zero() to use the Py_LONG_LONG casting thing.

PyBench contains a lot of different tests and executes them not too long. Random fluctuations of a few percent are possible. We need a more stable realistic test working with floats.

$ ./python -m timeit 'x,y,d=1,0,0.01
for i in range(628):
  x,y=x-d*y,y+d*x'

Without patch: 1000 loops, best of 3: 220 usec per loop

With patch: 1000 loops, best of 3: 232 usec per loop

Difference is 5%.

FWIW, I'm -1 on making the change. Any memory savings would be microscopic for most Python programs. And anything that slows down float creation (even a teeny bit) is detrimental to all Python programs. We're better off not adding special case logic unless there is a clear benefit (there isn't).

Why do you think it isn't safe, Antoine?
We only support IEEE platforms, and IEEE defines positive 0.0 to be all bits cleared.

I personally don't think that even a small measurable performance degradation in creating new floats is problematic since that is not where typical cpu power is spent. But this is the second time I´ve made this suggestion (the first was on python-dev) and in both cases I've run up against the opinion that it is a "rare" problem and worries about "performance." Yet this seemed not to be an issue when ints/longs were united.

So, while I was recently prompted to resubmit this suggestion, I feel no need to push the matter further and am just letting it rest here, then. I'm sure someone will think it is a good idea again in the future and can then reopen it.

Why do you think it isn't safe, Antoine?

It violates C's strict aliasing rules; Google for 'C strict aliasing' or 'C type punning' for more information. This isn't just a theoretical concern: gcc is known to make optimizations based on the assumption that strict aliasing isn't violated.

We only support IEEE platforms.

I don't think that's true, BTW.

return *(PY_LONG_LONG*)&fval == 0;
There is no aliasing, because there are no pointer variables in existence.
If we did this:

double *pfval = &fval;
PY_LONG_LONG *pl = (PY_LONG_LONG*)pfval
return *pfval == 0

Then we would have aliasing. Because "pfval" in this example doesn't exist but is merely a temporary, there is no aliasing.

As for IEEE compatibility, I don't think we could have our own floating point formatting library if we didn't make that assumption, but I might be wrong about that. On the other hand, I don't think there is a supported python architecture that defines positive zero as anything else than bitwise zero. And should such a platform be found, it is easy enough to disable this code for it.

Because "pfval" in this example doesn't exist but is merely a temporary, > there is no aliasing.

Maybe aliasing wasn't the right word to use, then. The exact rule being violated is C99 6.5, para. 7 (or C11 6.5 para. 7 if you prefer):

"""
An object shall have its stored value accessed only by an lvalue expression that has one of the following types: ...

"""

In any case, I don't think it's relevant whether the values involved are named variables or temporaries.

I don't think we could have our own floating point formatting library > if we didn't make that assumption

There are configure-time checks that determine whether that floating-point formatting library is used or not, including (amongst some other things) whether the underlying floating-point type is IEEE 754. Officially, Python shouldn't assume IEEE 754 floating-point anywhere---the core should build fine without it, and there's lots of code in the core to deal with non-IEEE 754 platforms.

And should such a platform be found, it is easy enough to disable this > code for it.

I think that's the wrong way around: the code should only be enabled in the first place when we're sure that the platform is IEEE 754. That shouldn't be hard to build into the patch, since the checks for IEEE 754 are already all over the place for other purposes.

Interesting.
I declare that this rule does not apply here since the code is a deliberate hack: We are pretending that a certain address points to integers and checking those integers.
If you insist on following the standard, you could do

double cmp = 0;
return mcmcmp(&cmp, &fval, sizeof(fval)) == 0;

but on all real machines this is the same as:

PY_LONG_LONG cmp = 0;
return mcmcmp(&cmp, &fval, sizeof(fval)) == 0;

Which again is the same as
return *(PY_LONG_LONG*)&fval == 0;
technically speaking, even if the standard doesn't agree. You could implement this with in-line assembly and so cheerfully sidestep the standard.

When you're hacking, your're hacking and the standard isn't your friend :)

As for IEEE, sure, anyway that thing is oriented is fine, although in this day and age, I find it rather amusing that the logic thinks of IEEE support as the exception, rather than the rule.

Anyway, this proposal has been rejected due to lack of interest or some misguided idea of performance, so the point is moot.

I declare that this rule does not apply here ...

Clearly the gcc developers disagree. :-)

iwasawa:~ mdickinson$ cat test2.c
int is_positive_zero(double f) {
  return *(long long*)&f == 0;
}
iwasawa:~ mdickinson$ gcc -fstrict-aliasing -O3 -Wall -Wextra -Wstrict-aliasing -c test2.c
test2.c: In function ‘is_positive_zero’:
test2.c:2: warning: dereferencing type-punned pointer will break strict-aliasing rules

Dang. I yield!