This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author christian.heimes
Recipients christian.heimes, terry.reedy
Date 2008-12-04.21:56:19
SpamBayes Score 0.00016984799
Marked as misclassified No
Message-id <1228427781.13.0.889197083898.issue4533@psf.upfronthosting.co.za>
In-reply-to
Content
The small buffer size in Modules/_fileio.c is one reason for the slowness.

$ dd if=/dev/zero of=zeros bs=1MB count=50
$ cat testread.py
open("zeros", "rb").read()
$ ./python -m cProfile testread.py
         40 function calls (39 primitive calls) in 4.246 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.016    0.016    4.246    4.246 <string>:1(<module>)     
        1    0.000    0.000    0.000    0.000 io.py:277(__new__)       
        2    0.000    0.000    0.000    0.000 io.py:355(flush)         
        2    0.000    0.000    0.000    0.000 io.py:364(close)         
        2    0.000    0.000    0.000    0.000 io.py:376(__del__)       
        1    0.000    0.000    0.000    0.000 io.py:413(_checkReadable)
        1    0.000    0.000    0.000    0.000 io.py:614(__init__)      
        2    0.000    0.000    0.000    0.000 io.py:618(close)         
        1    0.000    0.000    0.000    0.000 io.py:708(__init__)      
        1    0.000    0.000    0.000    0.000 io.py:733(flush)         
        1    0.000    0.000    0.000    0.000 io.py:736(close)         
        1    0.000    0.000    0.000    0.000 io.py:755(closed)        
        1    0.000    0.000    0.000    0.000 io.py:82(open)           
        1    0.000    0.000    0.000    0.000 io.py:896(__init__)      
        2    0.000    0.000    0.000    0.000 io.py:905(_reset_read_buf)
        1    0.021    0.021    4.230    4.230 io.py:909(read)           
        1    0.000    0.000    4.209    4.209 io.py:920(_read_unlocked) 
        1    0.000    0.000    0.000    0.000 {built-in method
allocate_lock}
      2/1    0.000    0.000    4.246    4.246 {built-in method exec}   
     
        1    0.000    0.000    0.000    0.000 {built-in method fstat}  
     
        2    0.000    0.000    0.000    0.000 {built-in method
isinstance}   
        3    0.000    0.000    0.000    0.000 {built-in method len}    
     
        1    0.000    0.000    0.000    0.000 {method '__enter__' of
'_thread.lock' objects}
        1    0.000    0.000    0.000    0.000 {method 'append' of 'list'
objects}           
        1    0.000    0.000    0.000    0.000 {method 'disable' of
'_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'fileno' of
'_FileIO' objects}          
        1    0.000    0.000    0.000    0.000 {method 'isatty' of
'_FileIO' objects}          
        1    0.825    0.825    0.825    0.825 {method 'join' of 'bytes'
objects}              
        2    3.384    1.692    3.384    1.692 {method 'read' of
'_FileIO' objects}            
        1    0.000    0.000    0.000    0.000 {method 'readable' of
'_FileIO' objects} 

$ vi Modules/_fileio.c
-#define DEFAULT_BUFFER_SIZE (8*1024)
+#define DEFAULT_BUFFER_SIZE (80*1024)
$ ./python -m cProfile testread.py                 
         40 function calls (39 primitive calls) in 1.273 CPU seconds   
            

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.019    0.019    1.273    1.273 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 io.py:277(__new__)
        2    0.000    0.000    0.000    0.000 io.py:355(flush)
        2    0.000    0.000    0.000    0.000 io.py:364(close)
        2    0.000    0.000    0.000    0.000 io.py:376(__del__)
        1    0.000    0.000    0.000    0.000 io.py:413(_checkReadable)
        1    0.000    0.000    0.000    0.000 io.py:614(__init__)
        2    0.000    0.000    0.000    0.000 io.py:618(close)
        1    0.000    0.000    0.000    0.000 io.py:708(__init__)
        1    0.000    0.000    0.000    0.000 io.py:733(flush)
        1    0.000    0.000    0.000    0.000 io.py:736(close)
        1    0.000    0.000    0.000    0.000 io.py:755(closed)
        1    0.000    0.000    0.000    0.000 io.py:82(open)
        1    0.000    0.000    0.000    0.000 io.py:896(__init__)
        2    0.000    0.000    0.000    0.000 io.py:905(_reset_read_buf)
        1    0.016    0.016    1.254    1.254 io.py:909(read)
        1    0.000    0.000    1.238    1.238 io.py:920(_read_unlocked)
        1    0.000    0.000    0.000    0.000 {built-in method
allocate_lock}
      2/1    0.000    0.000    1.273    1.273 {built-in method exec}
        1    0.000    0.000    0.000    0.000 {built-in method fstat}
        2    0.000    0.000    0.000    0.000 {built-in method isinstance}
        3    0.000    0.000    0.000    0.000 {built-in method len}
        1    0.000    0.000    0.000    0.000 {method '__enter__' of
'_thread.lock' objects}
        1    0.000    0.000    0.000    0.000 {method 'append' of 'list'
objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of
'_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'fileno' of
'_FileIO' objects}
        1    0.000    0.000    0.000    0.000 {method 'isatty' of
'_FileIO' objects}
        1    1.156    1.156    1.156    1.156 {method 'join' of 'bytes'
objects}
        2    0.081    0.041    0.081    0.041 {method 'read' of
'_FileIO' objects}
        1    0.000    0.000    0.000    0.000 {method 'readable' of
'_FileIO' objects}
History
Date User Action Args
2008-12-04 21:56:21christian.heimessetrecipients: + christian.heimes, terry.reedy
2008-12-04 21:56:21christian.heimessetmessageid: <1228427781.13.0.889197083898.issue4533@psf.upfronthosting.co.za>
2008-12-04 21:56:20christian.heimeslinkissue4533 messages
2008-12-04 21:56:19christian.heimescreate