classification
Title: Serialization of decimal.Decimal to XML-RPC
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: gdr, georg.brandl, mark.dickinson, rhettinger
Priority: normal Keywords:

Created on 2011-01-31 19:37 by gdr, last changed 2013-10-13 18:20 by georg.brandl. This issue is now closed.

Messages (7)
msg127651 - (view) Author: GDR! (gdr) Date: 2011-01-31 19:37
xmlrpc.client (and xmlrpclib in 2.x) can't serialize instances of Decimal, throwing TypeError instead. Because XML is a textual format, converting from decimal to float may cause loss of data.

According to http://www.xmlrpc.com/spec, encoding Decimal as XML-RPC <double /> is allowed: "The range of allowable values is implementation-dependent, is not specified." Therefore, including decimal numbers that can not be represented by IEEE double is allowed by the spec.

Also, in my opinion, making a numeric type available in standard library unsupported by another part of standard library, is counterintuitive, but it's just my personal point of view.

I can provide a patch if this bug report is considered proper.
msg127655 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-31 20:02
-1 on an implicit, lossy conversion.

The principal reasons for using decimal in the first place is avoid representation error.  For example, when money is being represented as a decimal, it is improper to convert it to float (where it can no longer be compared for equality and where the representable range is smaller).

A user needs explicit control in this situation, converting to float when it doesn't matter, or using an API that allows the decimal repr to be handled.

One other issue is that in an RPC environment, good design suggests that the client do its best to match the type expectations for of the functions on the server.  Implicit signature changing makes even less sense for RPC than it does in a regular application.
msg127660 - (view) Author: GDR! (gdr) Date: 2011-01-31 20:53
I didn't mean to implicitly convert Decimal to float. My point was that xmlrpclib should serialize Decimals natively to avoid loss of precision when converting to float and then to string.

Whether other party will be able to represent this number exactly or not should not be our issue. 

To make it more clear, this is what Python currently does:
http://gdr.pastebin.pl/35766
this is what I propose it shoud do:
http://gdr.pastebin.pl/35765
msg127661 - (view) Author: GDR! (gdr) Date: 2011-01-31 20:56
Also, I think that <double> should be represented as Decimal when unmarshalling XML-RPC data, because the standard doesn't give any limits on size on precision of data, and the only representation it allows is the one with decimal point, making it an excellent fit for Decimal.
msg127664 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-31 21:06
>>> x = decimal.Decimal('9999999999999.11111111111111')
>>> c.dumps(x,), 'whatever')
"<?xml version='1.0'?>\n<methodCall>\n<methodName>whatever</methodName>\n<params>\n<param>\n<value><double>9999999999999.11111111111111</double></value>\n</param>\n</params>\n</methodCall>\n"

That's what I though you meant and I don't think it is wise.  The RPC spec is all about interoperability.  The <double> tag means "double-precision signed floating point number" so what you're proposing an implicit type conversion (i.e. the client sends a decimal and the server receives a float).
msg127666 - (view) Author: GDR! (gdr) Date: 2011-01-31 21:24
While it would be allowed by the spec in the quotation I pasted in the original post, I understand your point, it makes sense. 

I still think, however, that it should be available at least as an option. There already is allow_none parameter available when marshaling. Another solution would be to allow something like json.JSONEncoder.
msg199744 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-13 18:20
Adding a special case for Decimal isn't as well justified as one for None -- after all it is a fundamental singleton. (Also, None is encoded without lossy conversion.)

The encoder/decoder are already extensible by extending Marshaller.dispatch and Unmarshaller.dispatch.
History
Date User Action Args
2013-10-13 18:20:28georg.brandlsetstatus: pending -> closed

nosy: + georg.brandl
messages: + msg199744

resolution: rejected
2013-05-25 14:51:11serhiy.storchakasetstatus: open -> pending
2011-02-01 09:01:08mark.dickinsonsetnosy: + mark.dickinson
2011-01-31 21:24:27gdrsetmessages: + msg127666
2011-01-31 21:06:32rhettingersetmessages: + msg127664
2011-01-31 20:56:15gdrsetmessages: + msg127661
2011-01-31 20:53:52gdrsetmessages: + msg127660
2011-01-31 20:02:15rhettingersetversions: + Python 3.3, - Python 3.1
nosy: + rhettinger

messages: + msg127655

type: behavior -> enhancement
2011-01-31 19:37:29gdrcreate