This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pitrou
Recipients alexandre.vassalotti, belopolsky, fdrake, mark.dickinson, pitrou, rhettinger
Date 2010-06-30.17:54:42
SpamBayes Score 0.0016149695
Marked as misclassified No
Message-id <1277920479.3240.8.camel@localhost.localdomain>
In-reply-to <1277919902.77.0.0519030541831.issue9120@psf.upfronthosting.co.za>
Content
> When talking about pickle speed performance, we need to take into
> account that most use cases for pickle are i/o bound.  IOW, the time
> to pickle and unpickle is dominated by the time needed to read/write
> the pickle to/from disk or to send it across the wire.  In those
> common use cases, any time spent compressing/decompressing is more
> than made for by savings in transport/storage time.

This is an oversimplification. Transport and storage performance is
mostly orthogonal to CPU performance; depending on the situation, you
might care about the former, or the latter, or both. Adding a
compression layer obviously taxes the CPU more, while optimizing
bytecode decreases CPU consumption as I've pointed out.

(for example, it is quite unlikely, IMO, that ZODB setups are I/O bound
rather than CPU bound, given that it's a database system written in
Python; and even optimized databases such as PostgreSQL can exhibit
CPU-bound behaviour, depending on the hardware configuration, the
database size and the queries issued)

> This is a fairly common speed optimization and it would be nice if
> pickles supported it directly (i.e. the sender specifies a compression
> option and the receiver automatically detects the option from an
> opcode in the pickle.

I'm not sure what would be the point. It is trivially easy for people to
add that compression layer to their own protocols if they want to. Also,
the C code for pickle doesn't have to be complicated by interfacing it
with various third-party compression libraries.

(a doc addition would be more reasonable)
History
Date User Action Args
2010-06-30 17:54:45pitrousetrecipients: + pitrou, fdrake, rhettinger, mark.dickinson, belopolsky, alexandre.vassalotti
2010-06-30 17:54:43pitroulinkissue9120 messages
2010-06-30 17:54:42pitroucreate