Here's a new version of the unicode reference type, extended
to run in both Python 2.6 and 3.1:
http://downloads.egenix.com/python/unicoderef-0.0.2.tar.gz
I've also included a benchmark implemented in C which measures
Unicode/Bytes allocation performance at high resolution
and without any VM overhead.
These are the results on a 64-bit system using a regular UCS2
build of Python 2.6.1 and 3.1 on a single core AMD system:
* Unicode allocation is always much slower than Bytes
This depends a lot on how much data you have to handle. If the
data fits into the pymalloc managed sizes, there's little
difference.
* CPU cache lines / memory boundaries obviously have some
impact on the results
If things are properly aligned, performance is better than
in the unaligned case. Unfortunately, there's nothing much the
CPython implementation can do to benefit from this, since the
hardware layouts are too diverse.
This explains why you see some unexpected figures in the
results, e.g. Python 3.1, 4096 words, 32chars - Unicode
being faster than Bytes.
* Something must have improved in pymalloc
The 4096 word tests perform better on 3.1 than on 2.6.
Since such improvement have a great impact, more
emphasis should be placed on these, e.g. by having pymalloc
provide more space to fixed size objects such PyUnicodeObject,
reducing the need to create new arenas.
Dictionaries and other PyObjects such as instances objects
would also benefit from such improvements.
Note that the benchmark scales the bytes word sizes to match
the internal allocation size of the Unicode objects, ie.
2chars maps to a length 4 bytes string on a UCS2 build.
Testing Unicode/Bytes Allocation Speed with Python 3.1rc1+
------------------------------------------------------------------------
=== 1024 words ============================================================
Variant 2chars, 1024 words, Unicode: 0.005503 seconds = 0.054 us per object
(best of 50 rounds)
Variant 2chars, 1024 words, Bytes : 0.006838 seconds = 0.067 us per object
(best of 50 rounds)
Variant 9chars, 1024 words, Unicode: 0.008956 seconds = 0.087 us per object
(best of 50 rounds)
Variant 9chars, 1024 words, Bytes : 0.007380 seconds = 0.072 us per object
(best of 50 rounds)
Variant 16chars, 1024 words, Unicode: 0.009257 seconds = 0.090 us per object
(best of 50 rounds)
Variant 16chars, 1024 words, Bytes : 0.007291 seconds = 0.071 us per object
(best of 50 rounds)
Variant 32chars, 1024 words, Unicode: 0.009589 seconds = 0.094 us per object
(best of 50 rounds)
Variant 32chars, 1024 words, Bytes : 0.007803 seconds = 0.076 us per object
(best of 50 rounds)
Variant pep100, 1024 words, Unicode: 0.010569 seconds = 0.103 us per object
(best of 50 rounds)
Variant pep100, 1024 words, Bytes : 0.008198 seconds = 0.080 us per object
(best of 50 rounds)
=== 2048 words ============================================================
Variant 2chars, 2048 words, Unicode: 0.011624 seconds = 0.057 us per object
(best of 50 rounds)
Variant 2chars, 2048 words, Bytes : 0.013941 seconds = 0.068 us per object
(best of 50 rounds)
Variant 9chars, 2048 words, Unicode: 0.018608 seconds = 0.091 us per object
(best of 50 rounds)
Variant 9chars, 2048 words, Bytes : 0.014773 seconds = 0.072 us per object
(best of 50 rounds)
Variant 16chars, 2048 words, Unicode: 0.018556 seconds = 0.091 us per object
(best of 50 rounds)
Variant 16chars, 2048 words, Bytes : 0.014550 seconds = 0.071 us per object
(best of 50 rounds)
Variant 32chars, 2048 words, Unicode: 0.018972 seconds = 0.093 us per object
(best of 50 rounds)
Variant 32chars, 2048 words, Bytes : 0.016377 seconds = 0.080 us per object
(best of 50 rounds)
Variant pep100, 2048 words, Unicode: 0.021005 seconds = 0.103 us per object
(best of 50 rounds)
Variant pep100, 2048 words, Bytes : 0.016636 seconds = 0.081 us per object
(best of 50 rounds)
=== 4096 words ============================================================
Variant 2chars, 4096 words, Unicode: 0.022950 seconds = 0.056 us per object
(best of 50 rounds)
Variant 2chars, 4096 words, Bytes : 0.027813 seconds = 0.068 us per object
(best of 50 rounds)
Variant 9chars, 4096 words, Unicode: 0.037229 seconds = 0.091 us per object
(best of 50 rounds)
Variant 9chars, 4096 words, Bytes : 0.031123 seconds = 0.076 us per object
(best of 50 rounds)
Variant 16chars, 4096 words, Unicode: 0.037118 seconds = 0.091 us per object
(best of 50 rounds)
Variant 16chars, 4096 words, Bytes : 0.036433 seconds = 0.089 us per object
(best of 50 rounds)
Variant 32chars, 4096 words, Unicode: 0.040970 seconds = 0.100 us per object
(best of 50 rounds)
Variant 32chars, 4096 words, Bytes : 0.051422 seconds = 0.126 us per object
(best of 50 rounds)
Variant pep100, 4096 words, Unicode: 0.049630 seconds = 0.121 us per object
(best of 50 rounds)
Variant pep100, 4096 words, Bytes : 0.034551 seconds = 0.084 us per object
(best of 50 rounds)
Testing Unicode/Bytes Allocation Speed with Python 2.6.1
------------------------------------------------------------------------
=== 1024 words ============================================================
Variant 2chars, 1024 words, Unicode: 0.005810 seconds = 0.057 us per object
(best of 50 rounds)
Variant 2chars, 1024 words, Bytes : 0.006918 seconds = 0.068 us per object
(best of 50 rounds)
Variant 9chars, 1024 words, Unicode: 0.008539 seconds = 0.083 us per object
(best of 50 rounds)
Variant 9chars, 1024 words, Bytes : 0.007618 seconds = 0.074 us per object
(best of 50 rounds)
Variant 16chars, 1024 words, Unicode: 0.008478 seconds = 0.083 us per object
(best of 50 rounds)
Variant 16chars, 1024 words, Bytes : 0.008097 seconds = 0.079 us per object
(best of 50 rounds)
Variant 32chars, 1024 words, Unicode: 0.008969 seconds = 0.088 us per object
(best of 50 rounds)
Variant 32chars, 1024 words, Bytes : 0.008649 seconds = 0.084 us per object
(best of 50 rounds)
Variant pep100, 1024 words, Unicode: 0.009779 seconds = 0.096 us per object
(best of 50 rounds)
Variant pep100, 1024 words, Bytes : 0.008318 seconds = 0.081 us per object
(best of 50 rounds)
=== 2048 words ============================================================
Variant 2chars, 2048 words, Unicode: 0.011745 seconds = 0.057 us per object
(best of 50 rounds)
Variant 2chars, 2048 words, Bytes : 0.013978 seconds = 0.068 us per object
(best of 50 rounds)
Variant 9chars, 2048 words, Unicode: 0.017418 seconds = 0.085 us per object
(best of 50 rounds)
Variant 9chars, 2048 words, Bytes : 0.014443 seconds = 0.071 us per object
(best of 50 rounds)
Variant 16chars, 2048 words, Unicode: 0.018138 seconds = 0.089 us per object
(best of 50 rounds)
Variant 16chars, 2048 words, Bytes : 0.016200 seconds = 0.079 us per object
(best of 50 rounds)
Variant 32chars, 2048 words, Unicode: 0.021348 seconds = 0.104 us per object
(best of 50 rounds)
Variant 32chars, 2048 words, Bytes : 0.018144 seconds = 0.089 us per object
(best of 50 rounds)
Variant pep100, 2048 words, Unicode: 0.019593 seconds = 0.096 us per object
(best of 50 rounds)
Variant pep100, 2048 words, Bytes : 0.016692 seconds = 0.082 us per object
(best of 50 rounds)
=== 4096 words ============================================================
Variant 2chars, 4096 words, Unicode: 0.024347 seconds = 0.059 us per object
(best of 50 rounds)
Variant 2chars, 4096 words, Bytes : 0.028411 seconds = 0.069 us per object
(best of 50 rounds)
Variant 9chars, 4096 words, Unicode: 0.041480 seconds = 0.101 us per object
(best of 50 rounds)
Variant 9chars, 4096 words, Bytes : 0.031532 seconds = 0.077 us per object
(best of 50 rounds)
Variant 16chars, 4096 words, Unicode: 0.047986 seconds = 0.117 us per object
(best of 50 rounds)
Variant 16chars, 4096 words, Bytes : 0.038006 seconds = 0.093 us per object
(best of 50 rounds)
Variant 32chars, 4096 words, Unicode: 0.062115 seconds = 0.152 us per object
(best of 50 rounds)
Variant 32chars, 4096 words, Bytes : 0.049687 seconds = 0.121 us per object
(best of 50 rounds)
Variant pep100, 4096 words, Unicode: 0.044331 seconds = 0.108 us per object
(best of 50 rounds)
Variant pep100, 4096 words, Bytes : 0.035939 seconds = 0.088 us per object
(best of 50 rounds) |
➜
