This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Robert.Withrow
Recipients Robert.Withrow, TD22057, loewis, mark.dickinson, vstinner
Date 2011-03-17.16:13:37
SpamBayes Score 0.0
Marked as misclassified No
Message-id <1300378418.0.0.0848432554192.issue4114@psf.upfronthosting.co.za>
In-reply-to
Content
Martin: in C I have the luxury of using 32 bit floats; not an option in Python.  Simple code doing the moral equivalent of NTOHL(HTONL()) works in this case for C but wouldn't help for Python.

Mark: I understand about the precision truncation issue and how Python does floating point arithmetic.  This C code clearly demonstrates what is going on:

#include <stdio.h>

int main(int argc, char *argv[])
{
  double d1 = 6.21;
  float f = 6.21;
  double d2 = f;
  
  printf("double: %.15f\n", d1);
  printf("float: %.15f\n", f);
  printf("double converted from float: %15.15f\n", d2);
}

The point here is about the contract of struct, NOT how Python does floating point arithmetic.  The contract is: what pack packs, unpack will unpack resulting in the original value.  At least, that is what the documentation leads you to believe.

For the 'f' format character, this contract is broken because of a basic implementation detail of Python and there is nothing in the documentation for struct that *directly* lets you know this will happen.  After all, the mentions in the documentation about 32 bit versus 64 bit talk about C not Python!

Even worse, there is no straightforward way (that I'm aware of) to write portable tests for code using the 'f' format character.  In my case I'm writing a tool that creates message codecs in multiple languages and the most basic unit test goes something like this:

m1 = example.message()
m1.f1 = 6.21
b = m.encode() # uses struct pack
m2 = example.message(b) # uses struct unpack
if m1 != m2:  # rich comparison
  print('fail')

This test will fail when you use the 'f' format code.

I suggest two things could be done to improve the situation:

1) Add a note to the documentation for struct that tells you that unpack(pack) using the 'f' format code will not generally give you the results you probably expect because <insert pointer to discussion of pythons use of C double versus C float here>.

2) Create a way in Python to write portable code related to 32 bit floats.  For example, if I had a way in Python to cause the precision truncation programmatically:

m1 = example.message()
m1.f1 = 6.21.as_32_bit_float() # Does the precision truncation upfront
b = m.encode() # uses struct pack
m2 = example.message(b) # uses struct unpack
if m1 != m2:  # rich comparison
  print('fail')

I'd expect this test to pass.

Hope this long-winded note helps.
History
Date User Action Args
2011-03-17 16:13:38Robert.Withrowsetrecipients: + Robert.Withrow, loewis, mark.dickinson, vstinner, TD22057
2011-03-17 16:13:38Robert.Withrowsetmessageid: <1300378418.0.0.0848432554192.issue4114@psf.upfronthosting.co.za>
2011-03-17 16:13:37Robert.Withrowlinkissue4114 messages
2011-03-17 16:13:37Robert.Withrowcreate