Author ymerej
Recipients ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, ymerej, zach.ware
Date 2021-07-19.15:49:31
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1626709771.89.0.929780396905.issue44675@roundup.psfhosted.org>
In-reply-to
Content
While writing a program using the multiprocessing library I stumbled upon what appears to be a bug with how different platforms deal with private methods.

When a class has a private method which is the target for a multiprocessing process, this name is correctly resolved on Linux (20.04.1-Ubuntu running Python 3.8.10) but fails to be resolved correctly on MacOS (Python 3.8.2 and 3.8.8) or Windows 10 (Python 3.9.6).


import multiprocessing

class Test(object):
    def __init__(self):
        self.a = 1
        self._b = 2
        self.__c = 3
        self.run1()
        self.run2()
    def _test1(self, conn):
        conn.send(self._b)
    def __test2(self, conn):
        conn.send(self.__c)
    def run1(self):
        print("Running self._test1()")
        parent, child = multiprocessing.Pipe()
        process = multiprocessing.Process(target=self._test1, args=(child, ))
        process.start()
        print(parent.recv())
        process.join()
    def run2(self):
        print("Running self.__test2()")
        parent, child = multiprocessing.Pipe()
        process = multiprocessing.Process(target=self.__test2, args=(child, ))
        process.start()
        print(parent.recv())
        process.join()

if __name__ == "__main__":
    t = Test()


On Linux, this has the intended behavior of printing:
Running self._test1()
2
Running self.__test2()
3

However, on Windows 10, this results in an Exception being raised:
Running self._test1()
2
Running self.__test2()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\<USER>\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\<USER>\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: 'Test' object has no attribute '__test2'

A similar exception is also raised on MacOS for this code.


It would therefore appear that there is different behavior for resolving class attributes starting with `__` on different platforms (at least within multiprocessing). It is my understanding that because multiprocessing.Process is called within the class, the private method should be within scope and so should resolve correctly.
I'm aware that Python doesn't have strict private methods, and instead renames them (Test.__test2 becomes Test._Test__test2) - explaining why on Windows it cannot find the attribute with that name. 

My question really is, which platform is correct here, and is the inconsistency intentional? I'd suggest Linux is most correct here as the process is spawned from within the object so the method should be in scope, but either way, the inconsistency between platforms may cause some unintended issues.
History
Date User Action Args
2021-07-19 15:49:31ymerejsetrecipients: + ymerej, paul.moore, ronaldoussoren, tim.golden, ned.deily, zach.ware, steve.dower
2021-07-19 15:49:31ymerejsetmessageid: <1626709771.89.0.929780396905.issue44675@roundup.psfhosted.org>
2021-07-19 15:49:31ymerejlinkissue44675 messages
2021-07-19 15:49:31ymerejcreate