I just ran the telco benchmark ...

  http://www.bytereef.org/mpdecimal/quickstart.html#telco-benchmark

... on _decimal to see how the PEP-393 changes affect the module.
The benchmark reads numbers from a binary file, does some calculations
and prints the result strings to a file.


Average results (10 iterations each):

Python 2.7:            5.87s
Revision 1726fa560112: 6.07s
Revision 7ffe3d304487: 6.56s


The bottleneck in telco.py is the line that writes a Decimal to the
output file:

  outfil.write("%s\n" % t)

The bottleneck in _decimal is (res is ascii):

   PyUnicode_FromString(res);

PyUnicode_DecodeASCII(res) has the same performance.


With this function ...

  static PyObject*
unicode_fromascii(const char* s, Py_ssize_t size)
{
    PyObject *res;
    res = PyUnicode_New(size, 127);
    if (!res)
        return NULL;
    memcpy(PyUnicode_1BYTE_DATA(res), s, size);
    return res;
}

... I get the same performance as with Python 2.7 (5.85s)!


I think it would be really beneficial for C-API users to have
more ascii low level functions that don't do error checking and
are simply as fast as possible.

Le 09/12/2011 22:12, Stefan Krah a écrit :
> The bottleneck in _decimal is (res is ascii):
>
>     PyUnicode_FromString(res);
>
> PyUnicode_DecodeASCII(res) has the same performance.
>
>
> With this function ...
>
>    static PyObject*
> unicode_fromascii(const char* s, Py_ssize_t size)
> {
>      PyObject *res;
>      res = PyUnicode_New(size, 127);
>      if (!res)
>          return NULL;
>      memcpy(PyUnicode_1BYTE_DATA(res), s, size);
>      return res;
> }
>
> ... I get the same performance as with Python 2.7 (5.85s)!

The problem is that  unicode_fromascii() is unsafe: it doesn't check 
that the string is pure ASCII. That's why this function is private.

Because of the PEP 383, ASCII and UTF-8 decoders (PyUnicode_DecodeASCII 
and PyUnicode_FromString) have to first scan the input to check for 
errors, and then do a fast memcpy. The scanner of these two decoders is 
already optimized to process the input string word by word (word=the C 
long type), instead of byte by byte, using a bit mask.

--

You can write your own super fast ASCII decoder using two lines:

     res = PyUnicode_New(size, 127);
     memcpy(PyUnicode_1BYTE_DATA(res), s, size);

(this is exactly what unicode_fromascii does)

> I think it would be really beneficial for C-API users to have
> more ascii low level functions that don't do error checking and
> are simply as fast as possible.

It is really important to ensure that a ASCII string doesn't contain 
characters outside [U+0000; U+007F] because many operations on ASCII 
string are optimized (e.g. UTF-8 pointer is shared with the ASCII pointer).

I prefer to not expose such function or someone will use it without 
understanding exactly how dangerous it is.

Martin and other may disagree with me.

Do you know Murphy's Law? :-)
http://en.wikipedia.org/wiki/Murphy%27s_law

> I prefer to not expose such function or someone will use it without
> understanding exactly how dangerous it is.

OK. - I'm afraid that I made an error in the benchmarks, since I
accidentally used a changed version of telco.py, namely:

    # t is a Decimal
    outfil.write("%s\n" % t) # original version
    ...

    outfil.write(str(t))     # changed version runs 
    outfil.write('\n')       # faster since PEP-393
    ...

Since PEP-393 the changed version with two calls to write()
runs quite a bit faster than the original. For Python-3.2
and 2.7 the original runs faster.

Do you have an idea what could cause this?

It's reasonable that string % formatting might have become slower...

I wonder what the issue is at this point. Unless you can state a clear issue that you want to see resolved, I propose to close this report as invalid.

Sorry, the title of the issue isn't correct any more. The revised
issue is that in 3.3

a) outfil.write("%s\n" % t)

is about 11% slower than in Python2.7 and 8% slower than in Python3.2.


On the other hand in 3.3 the hack

b) outfil.write(str(t)); outfil.write('\n') 

runs about as fast as a) in 3.2.


This doesn't necessarily show up in microbenchmarks with timeit, so
I thought I'd leave this open for others to see (and comment).

But if I understand correctly, the slowdown in string formatting is
expected, so we can indeed close this.

> But if I understand correctly, the slowdown in string formatting is
> expected, so we can indeed close this.

Well, expected doesn't mean it shouldn't be improved, so finding a way to speed it up would be nice ;)
(probably difficult, though)