This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eric.snow
Recipients Christian.Tismer, benjamin.peterson, eric.snow, methane, serhiy.storchaka, vstinner, yan12125
Date 2021-09-20.15:21:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1632151272.26.0.37855142602.issue34093@roundup.psfhosted.org>
In-reply-to
Content
FWIW, I found a faster solution than calling `w_object()` twice.

Currently the logic for w_ref() (used for each "complex" object) looks like this:

* if ob_ref == 1
   * do not apply FLAG_REF
   * marshal normally
* else if seen for the first time
   * apply FLAG_REF
   * marshal normally
* otherwise
   * emit TYPE_REF
   * emit the ref index of the first instance

The faster solution looks like this:

* if seen for the first time
   * do not apply FLAG_REF
   * marshal normally
   * record the index of the type byte in the output stream
* else if seen for a second time
   * apply FLAG_REF to the byte at the earlier-recorded position
   * emit TYPE_REF
   * emit the ref index of the first instance
* otherwise
   * emit TYPE_REF
   * emit the ref index of the first instance

While this is faster, there are two downsides: extra memory usage and it isn't practical when writing to a file.  However, I don't think either is a significant problem.  For the former, it can be mostly mitigated by using the negative values in WFILE.hashtable to store the type byte position.  For the latter, "marshal.dump()" is already a light wrapper around "marshal.dump()" and for PyMarshal_WriteObjectToFile() we simply stick with the current unstable approach (or change it to do what "marshal.dump()" does).

FYI, I mostly have that implemented in a branch, but am not sure when I'll get back to it.
History
Date User Action Args
2021-09-20 15:21:12eric.snowsetrecipients: + eric.snow, vstinner, benjamin.peterson, methane, serhiy.storchaka, Christian.Tismer, yan12125
2021-09-20 15:21:12eric.snowsetmessageid: <1632151272.26.0.37855142602.issue34093@roundup.psfhosted.org>
2021-09-20 15:21:12eric.snowlinkissue34093 messages
2021-09-20 15:21:12eric.snowcreate