Author tw.bert
Recipients tw.bert
Date 2014-06-22.16:01:10
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1403452871.25.0.30621640858.issue21826@psf.upfronthosting.co.za>
In-reply-to
Content
Preample: This is my first post to the python issue tracker, I included a fix.

This issue is probably related to http://bugs.python.org/issue11063 .

The problem:

After upgrading a package on AIX 7.1 x64 that started using the uuid module, we experienced serious performance issues.

The culprit (found after a day of debugging) is here:

File: ctypes/util.py
Note: The file /sbin/ldconfig does *not* exist, so no useful information is collected here.
The statement: 

`f = os.popen('/sbin/ldconfig -p 2>/dev/null')`

To be more specific about the performace at popen(), the performance degradation happens in it's close() method. It takes 300 ms, which is unacceptable. In a larger scope, statements that took 200ms now take 1400ms (because the above is called several times.

If I simply check for os.path.exists before the popen, the performance is fine again. See the included simple patch. It's a simple unix diff, we don't have git on that machine. Git can handle those diffs easily to my knowledge.

More information:

Small scope, culprit identified:

import os, time, traceback
print os.__file__
print time.clock(),'pre'
f = None
try:
  #if os.path.exists('/sbin/ldconfig'):
  f = os.popen('/sbin/ldconfig -p 2>/dev/null')
except:
  print traceback.format_exc()
finally:
  print time.clock(),'post close'
  if f: f.close()
  print time.clock(),'post finally'

This takes 300ms (post finally) without the check for exists.

Large scope, before patch:
time python -c "import hiredis;import redis;print 'redis-py version: %s , hiredis-py version: %s' %(redis.VERSION,hiredis.__ver
sion__,)"
redis-py version: (2, 10, 1) , hiredis-py version: 0.1.3

real    0m1.409s
user    0m0.416s
sys     0m0.129s

Large scope, after patch:
time python -c "import hiredis;import redis;print 'redis-py version: %s , hiredis-py version: %s' %(redis.VERSION,hiredis.__ver
sion__,)"
redis-py version: (2, 10, 1) , hiredis-py version: 0.1.3

real    0m0.192s
user    0m0.056s
sys     0m0.050s
History
Date User Action Args
2014-06-22 16:01:11tw.bertsetrecipients: + tw.bert
2014-06-22 16:01:11tw.bertsetmessageid: <1403452871.25.0.30621640858.issue21826@psf.upfronthosting.co.za>
2014-06-22 16:01:11tw.bertlinkissue21826 messages
2014-06-22 16:01:10tw.bertcreate