Message76934
The small buffer size in Modules/_fileio.c is one reason for the slowness.
$ dd if=/dev/zero of=zeros bs=1MB count=50
$ cat testread.py
open("zeros", "rb").read()
$ ./python -m cProfile testread.py
40 function calls (39 primitive calls) in 4.246 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.016 0.016 4.246 4.246 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 io.py:277(__new__)
2 0.000 0.000 0.000 0.000 io.py:355(flush)
2 0.000 0.000 0.000 0.000 io.py:364(close)
2 0.000 0.000 0.000 0.000 io.py:376(__del__)
1 0.000 0.000 0.000 0.000 io.py:413(_checkReadable)
1 0.000 0.000 0.000 0.000 io.py:614(__init__)
2 0.000 0.000 0.000 0.000 io.py:618(close)
1 0.000 0.000 0.000 0.000 io.py:708(__init__)
1 0.000 0.000 0.000 0.000 io.py:733(flush)
1 0.000 0.000 0.000 0.000 io.py:736(close)
1 0.000 0.000 0.000 0.000 io.py:755(closed)
1 0.000 0.000 0.000 0.000 io.py:82(open)
1 0.000 0.000 0.000 0.000 io.py:896(__init__)
2 0.000 0.000 0.000 0.000 io.py:905(_reset_read_buf)
1 0.021 0.021 4.230 4.230 io.py:909(read)
1 0.000 0.000 4.209 4.209 io.py:920(_read_unlocked)
1 0.000 0.000 0.000 0.000 {built-in method
allocate_lock}
2/1 0.000 0.000 4.246 4.246 {built-in method exec}
1 0.000 0.000 0.000 0.000 {built-in method fstat}
2 0.000 0.000 0.000 0.000 {built-in method
isinstance}
3 0.000 0.000 0.000 0.000 {built-in method len}
1 0.000 0.000 0.000 0.000 {method '__enter__' of
'_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'append' of 'list'
objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'fileno' of
'_FileIO' objects}
1 0.000 0.000 0.000 0.000 {method 'isatty' of
'_FileIO' objects}
1 0.825 0.825 0.825 0.825 {method 'join' of 'bytes'
objects}
2 3.384 1.692 3.384 1.692 {method 'read' of
'_FileIO' objects}
1 0.000 0.000 0.000 0.000 {method 'readable' of
'_FileIO' objects}
$ vi Modules/_fileio.c
-#define DEFAULT_BUFFER_SIZE (8*1024)
+#define DEFAULT_BUFFER_SIZE (80*1024)
$ ./python -m cProfile testread.py
40 function calls (39 primitive calls) in 1.273 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.019 0.019 1.273 1.273 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 io.py:277(__new__)
2 0.000 0.000 0.000 0.000 io.py:355(flush)
2 0.000 0.000 0.000 0.000 io.py:364(close)
2 0.000 0.000 0.000 0.000 io.py:376(__del__)
1 0.000 0.000 0.000 0.000 io.py:413(_checkReadable)
1 0.000 0.000 0.000 0.000 io.py:614(__init__)
2 0.000 0.000 0.000 0.000 io.py:618(close)
1 0.000 0.000 0.000 0.000 io.py:708(__init__)
1 0.000 0.000 0.000 0.000 io.py:733(flush)
1 0.000 0.000 0.000 0.000 io.py:736(close)
1 0.000 0.000 0.000 0.000 io.py:755(closed)
1 0.000 0.000 0.000 0.000 io.py:82(open)
1 0.000 0.000 0.000 0.000 io.py:896(__init__)
2 0.000 0.000 0.000 0.000 io.py:905(_reset_read_buf)
1 0.016 0.016 1.254 1.254 io.py:909(read)
1 0.000 0.000 1.238 1.238 io.py:920(_read_unlocked)
1 0.000 0.000 0.000 0.000 {built-in method
allocate_lock}
2/1 0.000 0.000 1.273 1.273 {built-in method exec}
1 0.000 0.000 0.000 0.000 {built-in method fstat}
2 0.000 0.000 0.000 0.000 {built-in method isinstance}
3 0.000 0.000 0.000 0.000 {built-in method len}
1 0.000 0.000 0.000 0.000 {method '__enter__' of
'_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'append' of 'list'
objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'fileno' of
'_FileIO' objects}
1 0.000 0.000 0.000 0.000 {method 'isatty' of
'_FileIO' objects}
1 1.156 1.156 1.156 1.156 {method 'join' of 'bytes'
objects}
2 0.081 0.041 0.081 0.041 {method 'read' of
'_FileIO' objects}
1 0.000 0.000 0.000 0.000 {method 'readable' of
'_FileIO' objects} |
|
Date |
User |
Action |
Args |
2008-12-04 21:56:21 | christian.heimes | set | recipients:
+ christian.heimes, terry.reedy |
2008-12-04 21:56:21 | christian.heimes | set | messageid: <1228427781.13.0.889197083898.issue4533@psf.upfronthosting.co.za> |
2008-12-04 21:56:20 | christian.heimes | link | issue4533 messages |
2008-12-04 21:56:19 | christian.heimes | create | |
|