classification
Title: Use pickle protocol 4 by default?
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: 33209 Superseder:
Assigned To: lukasz.langa Nosy List: Arfrever, alexandre.vassalotti, davin, josh.r, lukasz.langa, pitrou, sbt, serhiy.storchaka, vstinner
Priority: normal Keywords: patch, patch

Created on 2015-02-06 21:28 by serhiy.storchaka, last changed 2018-07-24 09:17 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6355 merged lukasz.langa, 2018-04-02 23:19
PR 6355 merged lukasz.langa, 2018-04-02 23:19
PR 6491 merged lukasz.langa, 2018-04-16 23:25
Messages (15)
msg235498 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-06 21:28
Why not use pickle protocol 4 by default? It allows to pickle some objects which are not pickleable with lower protocols and is more efficient.
msg235499 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-02-06 21:43
Because pickles will not be compatible with Python < 3.4 anymore. People who want maximum efficiency without compatibility can pass the right protocol manually.
msg235501 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-06 22:12
Multiprocessing uses default protocol and there is no simple (without hacking the stdlib) way to pass the right protocol manually.
msg235502 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-02-06 22:19
> Multiprocessing uses default protocol

AFAIK, this is so you can interact with processes using another version of Python.
msg235504 - (view) Author: Josh Rosenberg (josh.r) * Date: 2015-02-06 22:43
multiprocessing spawns the other processes itself from the same executable used to launch the main process. It's not subprocess. How would a different version of Python get involved?
msg235507 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-02-07 01:39
It is possible to have independent processes communicate together, although that's not the most widely-used feature. See:
https://docs.python.org/3/library/multiprocessing.html#multiprocessing-listeners-clients
Also:
https://docs.python.org/3/library/multiprocessing.html#managers
msg251858 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-29 13:40
Can't we "negociate" the protocol automatically?
msg275440 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-09-09 21:03
Closing in deference to the enhancement described in issue28053.
msg314841 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-04-02 23:07
Now that Python 3.3 is dead and 3.4 is soon to follow, it's safe to bump DEFAULT_PROTOCOL to 4.
msg314842 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-03 00:24
Should we bump the pickle protocol for shelve?
msg314927 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-04-04 06:06
New changeset c51d8c9ba6211d77db639487501f89aa9b4bcbb1 by Łukasz Langa in branch 'master':
bpo-23403: Bump pickle.DEFAULT_PROTOCOL to 4 (#6355)
https://github.com/python/cpython/commit/c51d8c9ba6211d77db639487501f89aa9b4bcbb1
msg315075 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-07 22:17
Is there something left to be done here?
msg315080 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-08 06:04
Shelve still uses protocol 3 by default. Should it be bumped too?
msg315117 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-09 08:54
> Shelve still uses protocol 3 by default. Should it be bumped too?

That sounds reasonable.  Perhaps open a separate issue for it?
msg322282 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-24 09:17
Opened issue34204 for shelve.
History
Date User Action Args
2018-07-24 09:17:14serhiy.storchakasetkeywords: patch, patch
status: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-07-24 09:17:01serhiy.storchakasetkeywords: patch, patch

messages: + msg322282
2018-04-16 23:25:29lukasz.langasetpull_requests: + pull_request6189
2018-04-09 08:54:43pitrousetkeywords: patch, patch

messages: + msg315117
2018-04-08 06:04:09serhiy.storchakasetkeywords: patch, patch

messages: + msg315080
2018-04-07 22:17:30pitrousetkeywords: patch, patch

messages: + msg315075
2018-04-04 06:06:56lukasz.langasetmessages: + msg314927
2018-04-03 01:08:54serhiy.storchakasetkeywords: patch, patch
dependencies: + Repeated Pickler.dump() doesn't reset the state in the C implementation of pickle
superseder: parameterize what serialization is used in multiprocessing ->
2018-04-03 00:24:39serhiy.storchakasetkeywords: patch, patch

messages: + msg314842
2018-04-02 23:19:21lukasz.langasetkeywords: + patch
stage: patch review
pull_requests: + pull_request6065
2018-04-02 23:19:19lukasz.langasetkeywords: + patch
stage: (no value)
pull_requests: + pull_request6064
2018-04-02 23:07:05lukasz.langasetversions: + Python 3.8, - Python 3.5
2018-04-02 23:07:01lukasz.langasetstatus: closed -> open
assignee: lukasz.langa
resolution: duplicate -> (no value)
messages: + msg314841
2018-04-02 22:38:27serhiy.storchakasetnosy: + lukasz.langa
2016-09-09 21:04:52davinsetsuperseder: parameterize what serialization is used in multiprocessing
2016-09-09 21:03:32davinsetstatus: open -> closed

nosy: + davin
messages: + msg275440

resolution: duplicate
2015-09-29 13:40:30vstinnersetnosy: + vstinner
messages: + msg251858
2015-02-07 04:15:31Arfreversetnosy: + Arfrever
2015-02-07 01:39:40pitrousetmessages: + msg235507
2015-02-06 22:43:28josh.rsetnosy: + josh.r
messages: + msg235504
2015-02-06 22:19:14pitrousetnosy: + sbt
messages: + msg235502
2015-02-06 22:12:31serhiy.storchakasetmessages: + msg235501
2015-02-06 21:43:51pitrousetmessages: + msg235499
2015-02-06 21:28:18serhiy.storchakacreate