Message166043
> It's under 64-bit Linux, Intel Core i5 CPU. Are you sure you're testing
> in non-debug mode?
I use 32-bit Linux.
> That said, the numbers under Windows suggest me that Eli's original idea
> (append and then join at the end) would be more robust.
I agree, it would be more robust, but much more complex solution. I will try to implement this approach too. Victor Stinner in their experiments with formatting preferred
overallocation approach (_PyUnicodeWriter), therefore, the solution probably will be a hybrid.
However, even in itself the patch deserves attention. It not only significant speeds up writing under Linux, but also speeds up the initialization, which leads to faster reading.
./python -m timeit -s "import io; d=(b'a'*99+b'\n')*10000" "s=io.BytesIO(d); r=s.readline" "while r(): pass"
Unpatched: 100 loops, best of 3: 5.92 msec per loop
Patched: 100 loops, best of 3: 3.95 msec per loop
I will try to preserve this advantage.
> By the way, here is Matt Mackall's take on realloc():
Note, that list also uses realloc() inside and therefore you get the same behavior with other constant factor. Actually, appending to a list less than sizeof(void*) bytes at a
time you will run into this sooner. Perhaps this is the cause that append/join approach under Windows slower than under Linux.
Thank you for measurements, this shows the scale of the issue. |
|
Date |
User |
Action |
Args |
2012-07-21 15:54:31 | serhiy.storchaka | set | recipients:
+ serhiy.storchaka, ncoghlan, pitrou, Arfrever, eli.bendersky, meador.inge, tshepang, jcon |
2012-07-21 15:54:30 | serhiy.storchaka | link | issue15381 messages |
2012-07-21 15:54:30 | serhiy.storchaka | create | |
|