Message402244
FWIW, I found a faster solution than calling `w_object()` twice.
Currently the logic for w_ref() (used for each "complex" object) looks like this:
* if ob_ref == 1
* do not apply FLAG_REF
* marshal normally
* else if seen for the first time
* apply FLAG_REF
* marshal normally
* otherwise
* emit TYPE_REF
* emit the ref index of the first instance
The faster solution looks like this:
* if seen for the first time
* do not apply FLAG_REF
* marshal normally
* record the index of the type byte in the output stream
* else if seen for a second time
* apply FLAG_REF to the byte at the earlier-recorded position
* emit TYPE_REF
* emit the ref index of the first instance
* otherwise
* emit TYPE_REF
* emit the ref index of the first instance
While this is faster, there are two downsides: extra memory usage and it isn't practical when writing to a file. However, I don't think either is a significant problem. For the former, it can be mostly mitigated by using the negative values in WFILE.hashtable to store the type byte position. For the latter, "marshal.dump()" is already a light wrapper around "marshal.dump()" and for PyMarshal_WriteObjectToFile() we simply stick with the current unstable approach (or change it to do what "marshal.dump()" does).
FYI, I mostly have that implemented in a branch, but am not sure when I'll get back to it. |
|
Date |
User |
Action |
Args |
2021-09-20 15:21:12 | eric.snow | set | recipients:
+ eric.snow, vstinner, benjamin.peterson, methane, serhiy.storchaka, Christian.Tismer, yan12125 |
2021-09-20 15:21:12 | eric.snow | set | messageid: <1632151272.26.0.37855142602.issue34093@roundup.psfhosted.org> |
2021-09-20 15:21:12 | eric.snow | link | issue34093 messages |
2021-09-20 15:21:12 | eric.snow | create | |
|