Message116173
It seems that multiprocessing is fairly conservative wrt. 64-bit support. For example, conn_send_string has a string limit of 0x7fffffff. Therefore, several of the warnings are harmless; the respective lengths fit into int just fine.
I recommend that the size_t parameters are converted to int in the places where this is actually known to be correct from control flow (e.g. both cases of conn_send_string), along with an assert() that the value being cast is <= INT_MAX.
IIUC, _conn_recvall can legitimately return int, no need to return Py_ssize_t: it will only return error codes (possibly MP_SUCCESS). Also, it might be sufficient to restrict the length of _conn_recvall to MAX_INT.
The alternate route would be to actually widen socket_connection to support 64 bit transmits (*); this would be Jesse's call. It's probably acceptable to defer this until somebody needs this enough to contribute a patch. Copying 2GiB over a pipe takes 1.5s on my system.
(*) My protocol proposal would be this: a length with the highest bit set (i.e. >= 0x80000000) indicates a huge message, where another four bytes of of length follow, for a total of 63 bits of length (MSB is unavailable as it serves as the length-of-length indicator). This would not waste bytes for small messages, and be backwards compatible. |
|
Date |
User |
Action |
Args |
2010-09-12 11:25:33 | loewis | set | recipients:
+ loewis, pitrou, tim.golden, jnoller, brian.curtin |
2010-09-12 11:25:32 | loewis | set | messageid: <1284290732.89.0.325557300982.issue9782@psf.upfronthosting.co.za> |
2010-09-12 11:25:31 | loewis | link | issue9782 messages |
2010-09-12 11:25:30 | loewis | create | |
|