Title: Optimizations for cgi.FieldStorage methods
Type: enhancement Stage:
Components: Library (Lib) Versions:
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: bkline, georg.brandl, jafo, jimjjewett
Priority: normal Keywords: patch

Created on 2006-08-16 17:37 by bkline, last changed 2007-09-20 16:06 by georg.brandl. This issue is now closed.

File name Uploaded Description Edit bkline, 2006-08-16 17:52 Patch for proposed optimization
Messages (11)
msg54877 - (view) Author: Bob Kline (bkline) * Date: 2006-08-16 17:37
Please add the following optimizations to the
FieldStorage class in

# =================================================

    def keys(self):
        """Dictionary style keys() method."""
        if self.list is None:
            raise TypeError, "not indexable"
        return {}.fromkeys([ for item in

    def __nonzero__(self):
        """Support for efficient test of instance"""
        return self.list and True or False

# =================================================

The __nonzero__ method is new, and keys() method is a
replacement for code which built the list of unique
fields names by hand, and which took several orders of
magnitude longer to perform.

If you need me to post this as a patch against a
certain version, let me know.
msg54878 - (view) Author: Bob Kline (bkline) * Date: 2006-08-16 17:52
Logged In: YES 

I didn't realize this interface would strip leading blanks
from lines.  Patch submitted to get around this problem.
msg54879 - (view) Author: Jim Jewett (jimjjewett) Date: 2006-08-17 22:52
Logged In: YES 

Why are you creating a dictionary just to take its keys?

Is there a reason the keys method can't just return

[ for item in self.list] 

msg54880 - (view) Author: Bob Kline (bkline) * Date: 2006-08-17 23:55
Logged In: YES 

> Why are you creating a dictionary just to take its keys?

Because in order to preserve the original behavior (and to
match the documentation), the list returned must contain
only one member for each unique field name.  The built-in
set() would work just as well, though my benchmarks don't
show much difference in performance.  Something like:

return list(set([ for i in self.list]))
msg56032 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2007-09-19 08:12
Georg: This has been assigned to you, do you have any thoughts on this?
 If you don't, please assign back to me.
msg56034 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-09-19 09:19
Thanks for notifying me. The tracker currently doesn't send
notifications if you only set the assignee, but don't include a message.
I hope this will be fixed soon.

The __nonzero__() is unproblematic. The keys() produced in this way will
have an unpredictable order, while the original way preserves the order
of names in self.list. I don't know whether that would cause problems.
msg56035 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2007-09-19 09:27
Thanks for correcting my mis-click on the "patch" item, gbot.

As far as the order goes, is this something that needs to be discussed
on c.l.p, or should we assign it to someone else who might have an
opinion on it?
msg56038 - (view) Author: Bob Kline (bkline) * Date: 2007-09-19 14:40
Please note that the documentation of the keys() method of the
FieldStorage class (both in the method's docstring as well as in the
separate library manual) describes the method as a "dictionary style
keys() method."  Section 3.8 of the documentation has this to say about
the keys() method of a dictionary: "Keys and values are listed in an
arbitrary order which is non-random, varies across Python
implementations, and depends on the dictionary's history of insertions
and deletions."  I believe the proposed implementation conforms with
this specified behavior.
msg56039 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-09-19 14:45
While this is true, there may be code relying on the current behavior,
and to break that for a tiny performance gain is gratuitous breakage and
should be avoided.
msg56040 - (view) Author: Bob Kline (bkline) * Date: 2007-09-19 15:20
I'm not sure I would characterize a speedup of several orders of
magnitude a "tiny performance gain."  We had scripts with very large
numbers of fields which were actually timing out.  While I understand
and agree with the principle of breaking as little existing code as
possible, when programmers have actually been told that the method
behaves the way the dictionary keys() method behaves it seems
unreasonable to assume that the method will preserve the original order
of fields (whatever that might mean for multiply-occurring field names).
msg56055 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-09-20 16:06
Okay, you've convinced me. Committed rev. 58218.
Date User Action Args
2007-09-20 16:06:32georg.brandlsetstatus: open -> closed
resolution: accepted
messages: + msg56055
2007-09-19 15:20:58bklinesetmessages: + msg56040
2007-09-19 14:45:48georg.brandlsetmessages: + msg56039
2007-09-19 14:40:22bklinesetmessages: + msg56038
2007-09-19 09:27:24jafosetmessages: + msg56035
2007-09-19 09:19:36georg.brandlsetkeywords: + patch, - py3k
messages: + msg56034
2007-09-19 08:12:00jafosetkeywords: + py3k
nosy: + jafo
messages: + msg56032
2006-08-16 17:37:16bklinecreate