This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author poostenr
Recipients eric.smith, poostenr, ubehera
Date 2016-01-14.23:55:40
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1452815740.73.0.365081021544.issue26118@psf.upfronthosting.co.za>
In-reply-to
Content
My initial observations with my Python script using:

s = "{0},".format(columnvalue)   # fast
Processed ~360MB of data from 2:16PM - 2:51PM (35 minutes, ~10MB/min)
One particular file 6.5MB took ~1 minute.

When I changed this line of code to:
s = "'{0}',".format(columnvalue) # ~30x slower (1 min. vs 30 min.)
Same particular file of 6.5MB took ~30 minutes (228KB/min).

My Python environment is:
C:\Data>python -V
Python 3.5.1 :: Anaconda 2.4.1 (32-bit)

I did some more testing with a very simplified piece of code, but is not conclusive. But there is a significant jump when I introduce the single quotes where I see a jump from 0m2.410s to 0m3.875s.

$ python -V
Python 3.5.1

// 
// s='test'
// for x in range(10000000):
//     y = "{0}".format(s)

$ time python test.py

real	0m2.410s
user	0m2.356s
sys	0m0.048s

// s='test'
// for x in range(10000000):
//     y = "'%s'" % (s)

$ time python test2.py 

real	0m2.510s
user	0m2.453s
sys	0m0.051s

// s='test'
// for x in range(10000000):
//     y = "'{0}'".format(s)

$ time python test3.py 

real	0m3.875s
user	0m3.819s
sys	0m0.048s
History
Date User Action Args
2016-01-14 23:55:40poostenrsetrecipients: + poostenr, eric.smith, ubehera
2016-01-14 23:55:40poostenrsetmessageid: <1452815740.73.0.365081021544.issue26118@psf.upfronthosting.co.za>
2016-01-14 23:55:40poostenrlinkissue26118 messages
2016-01-14 23:55:40poostenrcreate