➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-11-03 08:00 by ahcub, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| error_on_bug_multiprocessing_output.py | ahcub, 2018-11-03 08:00 | |||
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 10305 | open | ahcub, 2018-11-03 08:03 | |
| Messages (2) | |||
|---|---|---|---|
| msg329189 - (view) | Author: Oleksandr Buchkovskyi (ahcub) * | Date: 2018-11-03 08:00 | |
the problem is reproduced on big multiprocessing.Process output
when the size of the output gets bigger than signed int a struct error is raised
```
python3 test.py
Process ForkPoolWorker-1:
Traceback (most recent call last):
File "/usr/lib64/python3.5/multiprocessing/pool.py", line 125, in worker
put((job, i, result))
File "/usr/lib64/python3.5/multiprocessing/queues.py", line 347, in put
self._writer.send_bytes(obj)
File "/usr/lib64/python3.5/multiprocessing/connection.py", line 200, in send_bytes
self._send_bytes(m[offset:offset + size])
File "/usr/lib64/python3.5/multiprocessing/connection.py", line 393, in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.5/multiprocessing/process.py", line 252, in _bootstrap
self.run()
File "/usr/lib64/python3.5/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.5/multiprocessing/pool.py", line 130, in worker
put((job, i, (False, wrapped)))
File "/usr/lib64/python3.5/multiprocessing/queues.py", line 347, in put
self._writer.send_bytes(obj)
File "/usr/lib64/python3.5/multiprocessing/connection.py", line 200, in send_bytes
self._send_bytes(m[offset:offset + size])
File "/usr/lib64/python3.5/multiprocessing/connection.py", line 393, in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
```
this error can be fixed by changing types that are used herу for struct.pack/unpack to !Q
|
|||
| msg329190 - (view) | Author: Karthikeyan Singaravelan (xtreak) *  |
Date: 2018-11-03 08:17 | |
Thanks for the report and PR. I think these are related issues : issue17560, issue28506 with issue17560 being open one. Correct me if I am wrong here. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:07 | admin | set | github: 79333 |
| 2018-11-03 08:21:36 | serhiy.storchaka | set | status: open -> closed superseder: problem using multiprocessing with really big objects? resolution: duplicate stage: patch review -> resolved |
| 2018-11-03 08:17:28 | xtreak | set | nosy:
+ xtreak messages: + msg329190 |
| 2018-11-03 08:03:28 | ahcub | set | keywords:
+ patch stage: patch review pull_requests: + pull_request9613 |
| 2018-11-03 08:00:06 | ahcub | create | |