Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why not drop the _active list? #43107

Closed
hvbtup mannequin opened this issue Mar 29, 2006 · 11 comments
Closed

Why not drop the _active list? #43107

hvbtup mannequin opened this issue Mar 29, 2006 · 11 comments

Comments

@hvbtup
Copy link
Mannequin

hvbtup mannequin commented Mar 29, 2006

BPO 1460493
Nosy @loewis

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2006-04-10.15:58:24.000>
created_at = <Date 2006-03-29.07:16:44.000>
labels = []
title = 'Why not drop the _active list?'
updated_at = <Date 2006-04-10.15:58:24.000>
user = 'https://bugs.python.org/hvbtup'

bugs.python.org fields:

activity = <Date 2006-04-10.15:58:24.000>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['None']
creation = <Date 2006-03-29.07:16:44.000>
creator = 'hvb_tup'
dependencies = []
files = []
hgrepos = []
issue_num = 1460493
keywords = []
message_count = 11.0
messages = ['27934', '27935', '27936', '27937', '27938', '27939', '27940', '27941', '27942', '27943', '27944']
nosy_count = 5.0
nosy_names = ['loewis', 'nnorwitz', 'atila-cheops', 'hvb_tup', 'tfaujour']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1460493'
versions = []

@hvbtup
Copy link
Mannequin Author

hvbtup mannequin commented Mar 29, 2006

I am using a modified version of subprocess.py,

where I have removed the _active list and all
references to it.

I have tested it (under Windows 2000) and there were
no errors.

So what is the reason for managing the _active list
at all? Why not drop it?

@hvbtup hvbtup mannequin closed this as completed Mar 29, 2006
@tfaujour
Copy link
Mannequin

tfaujour mannequin commented Mar 29, 2006

Logged In: YES
user_id=1488657

I agree.

The use of _active makes subprocess.py thread-UNsafe.

See also: Bug bpo-1199282

In order to have a thread-safe subprocess.py, I commented
out the call to _cleanup() in Popen.__init__(). As a side
effect, _active becomes useless.

@loewis
Copy link
Mannequin

loewis mannequin commented Mar 29, 2006

Logged In: YES
user_id=21627

The purpose of the _active list is to wait(2) for open
processes. It needs to stay.

@nnorwitz
Copy link
Mannequin

nnorwitz mannequin commented Mar 30, 2006

Logged In: YES
user_id=33168

If you always called wait() the _active list isn't
beneficial to you. However, many people do not call wait
and the _active list provides a mechanism to cleanup zombied
children. This is important for many users.

If you need thread safely, you can handle the locking
yourself before calling poll()/wait().

@atila-cheops
Copy link
Mannequin

atila-cheops mannequin commented Mar 30, 2006

Logged In: YES
user_id=1276121

what happens if you are doing a _cleanup (iterating over a
copy of _active) in multiple threads?
can it not happen then that you clean up a process 2 times?

thread 1 starts a _cleanup: makes a copy of _active[:] and
starts polling
thread 2 starts a _cleanup: makes a copy of _active[:] and
starts polling

thread 1 encounters a finished process and removes it from
_active[]
thread 2 does not know the process is removed, finds the
same process finished and tries to remove it from _active
but this fails, because thread 1 removed it already

so the action of cleaning up should maybe be serialized
if 1 thread is doing it, the other one should block

everyone who needs this can of course patch the
subprocess.py file, but shouldn't this be fixed in the library?

@atila-cheops
Copy link
Mannequin

atila-cheops mannequin commented Mar 30, 2006

Logged In: YES
user_id=1276121

the same problem probably exists in popen2.py
there _active is also used
so if something is fixed in subprocess.py, it should
probably also be fixed in popen2.py

@atila-cheops
Copy link
Mannequin

atila-cheops mannequin commented Mar 30, 2006

Logged In: YES
user_id=1276121

also bpo-1214859 is interesting, has a patch

@loewis
Copy link
Mannequin

loewis mannequin commented Mar 30, 2006

Logged In: YES
user_id=21627

attila-cheops, please read the 2.5 version of popen2.

popen2 now only adds processes to _active in __del__, not in
__init__, so the problem with the application still wanting
to wait/poll is solved.

Multiple threads simultaneously isn't a problem, since it is
(or should be) harmless to invoke poll on a process that has
already been waited for. For only one of the poll calls, the
wait will actually succeed, and that should be the one that
removes it from the _active list.

The same strategy should be applied to subprocess.

@atila-cheops
Copy link
Mannequin

atila-cheops mannequin commented Apr 5, 2006

Logged In: YES
user_id=1276121

the implementation in 2.5 of popen2 seems good
if you or astrand could patch subprocess.py accordingly,
that would be great.

@atila-cheops
Copy link
Mannequin

atila-cheops mannequin commented Apr 10, 2006

Logged In: YES
user_id=1276121

see patch bpo-1467770

@loewis
Copy link
Mannequin

loewis mannequin commented Apr 10, 2006

Logged In: YES
user_id=21627

This was fixed with said patch.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants