This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Promise the long-time truth that `args=list` works
Type: behavior Stage: patch review
Components: Documentation Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: CharlieZhao, docs@python, rhettinger, serhiy.storchaka, tim.peters
Priority: low Keywords: easy, patch

Created on 2021-11-06 04:02 by tim.peters, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 29437 closed rhettinger, 2021-11-06 05:01
PR 30982 merged CharlieZhao, 2022-01-28 07:57
Messages (7)
msg405846 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2021-11-06 04:02
A number of contexts allow specifying a tuple of arguments to be passed later to a function. The Thread constructor is a fine example, and happened to come up (again! for me) here today:

https://stackoverflow.com/questions/69858950/why-do-we-have-to-add-comma-in-args-in-python-multithreading/69859068

This often confuses especially newbies, because the function they intend to parallelize often takes only a single argument, and Python's syntax for a 1-element tuple actually _requires_ parentheses in the context of an argument list, with a naked trailing comma:

t = threading.Thread(target=access, args=(thread_number,))

It "looks weird" to people.

I'm not suggesting to change that, but instead to officially bless the workaround I've seen very often in real code: use a list instead.

t = threading.Thread(target=access, args=[thread_number])

Nobody scratches their head over what that means.

CPython's implementations typically couldn't care less what kind of sequence is used, and none that I'm aware of verify that it's specifically a tuple. The implementations just go on to do some simple variation of

    self.target(*self.args)

Tuple or list makes no real difference. I'm not really keen to immortalize the "any sequence type whatsoever that just happens to work" implementation behavior, but am keen to promise that a list specifically will work. A lot of code already relies on it.
msg405851 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-11-06 09:09
There is a difference if you modify the arguments list after creating a thread.

    args = [1]
    t = threading.Thread(target=access, args=args)
    args[0] = 2
    t.start()

Would it call access(1) or access(2)?
msg405867 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2021-11-06 16:58
Serhiy, we haven't documented such stuff, and, indeed, I've been burned by it but much more often in the case of multiprocessing.Process. But note that I'm SWAPPING the order of your last two lines. In the original, you mutated the argument _before_ starting any parallel work, so "of course" the new worker will see the mutation:

    def access(xs):
        print(xs)

    args = ([1],)
    t = multiprocessing.Process(target=access, args=args)
    t.start() # start parallel work before mutating
    args[0][0] = 2



Does that print [1] or [2]? Passing a tuple in no way prevents mutations to mutable objects the tuple contains.

When the docs are silent, "implementation defined" rules. Whether you use threading or multiprocessing in the altered example above, the result printed simply isn't defined - it's a race between the main thread doing the mutation and the "parallel part" accessing the mutated object.

This is subtler in the multiprocessing context, though, because the relevant "parallel part" is really the hidden thread that pickles the argument list to send to the worker. That effectively makes a deep copy. But it's still a race, just not one visible from staring at the Python code. In the threading case, no copies are made.
msg406145 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2021-11-11 03:22
Changed stage back to "needs patch", since Raymond appears to have closed his PR. Raymond, what's up with that?
msg411823 - (view) Author: Charlie Zhao (CharlieZhao) * Date: 2022-01-27 02:26
I'd like to work on this issue recently if it's still needed.

According to suggestions in PR:https://github.com/python/cpython/pull/29437, I will:

* Add doc example for Thread function;
* Add some test cases for checking the validity of list args;
* Repeat the above works on multiprocessing module.
msg411825 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2022-01-27 02:43
Charliz, please do! I have no idea why Raymond just stopped. He even deleted his initial message here, saying "I relied on this for many years.  So, yet it would be nice to guarantee it :-)".

Best I can tell, nothing has changed: lots of people have relied on this behavior for years, and the docs should say it's fine.
msg414067 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2022-02-26 04:17
New changeset e466faa9df9a1bd377d9725de5484471bc4af8d0 by Charlie Zhao in branch 'main':
bpo-45735: Promise the long-time truth that `args=list` works (GH-30982)
https://github.com/python/cpython/commit/e466faa9df9a1bd377d9725de5484471bc4af8d0
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89898
2022-02-26 04:17:24tim.peterssetmessages: + msg414067
2022-01-28 07:57:55CharlieZhaosetstage: needs patch -> patch review
pull_requests: + pull_request29161
2022-01-27 02:43:08tim.peterssetmessages: + msg411825
2022-01-27 02:26:42CharlieZhaosetnosy: + CharlieZhao
messages: + msg411823
2021-11-11 03:22:04tim.peterssetmessages: + msg406145
stage: patch review -> needs patch
2021-11-06 16:58:17tim.peterssetmessages: + msg405867
2021-11-06 09:09:32serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg405851
2021-11-06 05:12:19rhettingersetmessages: - msg405847
2021-11-06 05:01:09rhettingersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request27691
2021-11-06 04:44:44rhettingersetnosy: + rhettinger
messages: + msg405847
2021-11-06 04:02:32tim.peterssettitle: Promise that the long-time truth that `args=list` works -> Promise the long-time truth that `args=list` works
2021-11-06 04:02:01tim.peterscreate