Message117372
> > Well, first, this would only work for large objects. [...]
> > Why do you think you might have such duplication in your workload?
>
> Some of the projects with which I work involve multiple manipulations
> of large datasets. Often, we use Python scripts as "first and third"
> stages in a pipeline. For example, in one current workflow, we read a
> large file into a cStringIO object, do a few manipulations with it,
> pass it off to a second process, and await the results.
Why do you read it into a cStringIO? A cStringIO has the same interface
as a file, so you could simply operate on the file directly.
(you could also try mmap if you need quick random access to various
portions of the file) |
|
Date |
User |
Action |
Args |
2010-09-25 14:31:37 | pitrou | set | recipients:
+ pitrou, georg.brandl, amaury.forgeotdarc, hunteke |
2010-09-25 14:31:35 | pitrou | link | issue9942 messages |
2010-09-25 14:31:35 | pitrou | create | |
|