Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string #88118

anon01 · 2021-04-27T11:45:16Z

BPO	43952
Nosy	@miguendes
PRs	bpo-43952: Fix multiprocessing Listener authkey bug #25845

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-04-27.11:45:15.510>
labels = ['type-bug', 'library', '3.11']
title = 'Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string'
updated_at = <Date 2021-05-13.21:12:02.688>
user = 'https://bugs.python.org/anon01'

bugs.python.org fields:

activity = <Date 2021-05-13.21:12:02.688>
actor = 'pitrou'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-04-27.11:45:15.510>
creator = 'anon01'
dependencies = []
files = []
hgrepos = []
issue_num = 43952
keywords = ['patch']
message_count = 3.0
messages = ['392056', '392591', '392790']
nosy_count = 1.0
nosy_names = ['miguendes']
pr_nums = ['25845']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43952'
versions = ['Python 3.11']

Linked PRs

anon01 · 2021-04-27T11:45:15Z

When I run this code (on a UNIX system with temporary directory /tmp/):

from multiprocessing import connection
import threading
key=b"1"

def run():
    c=connection.Client("/tmp/xxx", authkey=key)
    c.send("data")

l=connection.Listener("/tmp/xxx", authkey=key)
threading.Thread(target=run).start()
c=l.accept()
print("receive", c.recv())

it prints out "receive data" normally.

However, in the special case that key=b"" the doesn't print anything, and can only be interrupted with Ctrl+C.

key=None doesn't have that issue.

Note that this issue does happen when the client uses the key b"" and the server uses the key None, but the program works normally if the reverse situation.

Python version: Python 3.9.3.

miguendes · 2021-05-01T12:48:54Z

I tried debugging this and from what I can see it's because there's an if that checks if the authkey is not None in the Client constructor:

https://github.com/python/cpython/blob/v3.9.4/Lib/multiprocessing/connection.py#L512

    if authkey is not None:
        answer_challenge(c, authkey)
        deliver_challenge(c, authkey)

Whereas in the Listener, the check is different:

https://github.com/python/cpython/blob/v3.9.4/Lib/multiprocessing/connection.py#L469

        c = self._listener.accept()
        if self._authkey:
            deliver_challenge(c, self._authkey)
            answer_challenge(c, self._authkey)
        return c

If I change the Listener to:

        if self._authkey is not None:
            deliver_challenge(c, self._authkey)
            answer_challenge(c, self._authkey)
        return c

it works.

The docs say:

"""
If authkey is given and not None, it should be a byte string and will be used as the secret key for an HMAC-based authentication challenge. No authentication is done if authkey is None. AuthenticationError is raised if authentication fails. See Authentication keys.
"""

Now the question is, if None is OK because no auth will be done what about empty bytes? Can it be used as secret key? If empty bytes is not accepted shouldn't Listener/Client raise an exception in the constructor?

miguendes · 2021-05-03T10:21:39Z

I had a look at the HMAC RFC and apparently empty bytes sequence can be used as secret key.

"The definition of HMAC requires a cryptographic hash function, which
we denote by H, and a secret key K.

...

The authentication key K can be of any length up to B, the
block length of the hash function."

https://tools.ietf.org/html/rfc2104.html#section-2

Assuming that is the case, the fix would be to change the Listener to:

        if self._authkey is not None:
            deliver_challenge(c, self._authkey)
            answer_challenge(c, self._authkey)
        return c

I created a PR for that, if anyone can review it, I appreciate it.
#25845

gpshead · 2024-02-27T16:44:21Z

A bit of an ironic issue as using an empty key seems kind of pointless... but thanks for the edge case fix! :)

Fix some test_multiprocessing flakiness. Potentially introduced by #25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ```

…6434) Fix some test_multiprocessing flakiness. Potentially introduced by python#25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ``` (cherry picked from commit ea1803e) Co-authored-by: Gregory P. Smith <greg@krypto.org>

…GH-116440) Fix some test_multiprocessing flakiness. Potentially introduced by #25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ``` (cherry picked from commit ea1803e) Co-authored-by: Gregory P. Smith <greg@krypto.org>

…GH-116441) Fix some test_multiprocessing flakiness. Potentially introduced by #25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ``` (cherry picked from commit ea1803e) Co-authored-by: Gregory P. Smith <greg@krypto.org>

Fix some test_multiprocessing flakiness. Potentially introduced by python#25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ```

anon01 mannequin added 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Apr 27, 2021

pitrou added 3.11 only security fixes and removed 3.9 only security fixes labels May 13, 2021

ezio-melotti transferred this issue from another repository Apr 10, 2022

vstinner added the topic-multiprocessing label Jun 6, 2022

serhiy-storchaka closed this as completed Feb 27, 2024

bedevere-app bot mentioned this issue Mar 6, 2024

gh-88118: Fix some test_multiprocessing flakiness. #116434

Merged

This was referenced Mar 6, 2024

[3.12] gh-88118: Fix some test_multiprocessing flakiness. (GH-116434) #116440

Merged

[3.11] gh-88118: Fix some test_multiprocessing flakiness. (GH-116434) #116441

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string #88118

Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string #88118

anon01 mannequin commented Apr 27, 2021 •

edited by bedevere-app bot

anon01 mannequin commented Apr 27, 2021

miguendes mannequin commented May 1, 2021

miguendes mannequin commented May 3, 2021

gpshead commented Feb 27, 2024

Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string #88118

Multiprocessing UNIX socket connection: client freeze if authkey is an empty byte string #88118

Comments

anon01 mannequin commented Apr 27, 2021 • edited by bedevere-app bot

Linked PRs

anon01 mannequin commented Apr 27, 2021

miguendes mannequin commented May 1, 2021

miguendes mannequin commented May 3, 2021

gpshead commented Feb 27, 2024

anon01 mannequin commented Apr 27, 2021 •

edited by bedevere-app bot