Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiprocessing creates incorrect pids #59808

Closed
cfriedline mannequin opened this issue Aug 9, 2012 · 7 comments
Closed

Multiprocessing creates incorrect pids #59808

cfriedline mannequin opened this issue Aug 9, 2012 · 7 comments
Labels
type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@cfriedline
Copy link
Mannequin

cfriedline mannequin commented Aug 9, 2012

BPO 15603
Nosy @pitrou

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-08-09.20:20:36.711>
created_at = <Date 2012-08-09.15:03:18.909>
labels = ['type-crash']
title = 'Multiprocessing creates incorrect pids'
updated_at = <Date 2012-08-09.20:20:36.709>
user = 'https://bugs.python.org/cfriedline'

bugs.python.org fields:

activity = <Date 2012-08-09.20:20:36.709>
actor = 'cfriedline'
assignee = 'none'
closed = True
closed_date = <Date 2012-08-09.20:20:36.711>
closer = 'cfriedline'
components = []
creation = <Date 2012-08-09.15:03:18.909>
creator = 'cfriedline'
dependencies = []
files = []
hgrepos = []
issue_num = 15603
keywords = []
message_count = 7.0
messages = ['167788', '167793', '167797', '167798', '167799', '167801', '167830']
nosy_count = 2.0
nosy_names = ['pitrou', 'cfriedline']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue15603'
versions = ['Python 2.7']
@cfriedline
Copy link
Mannequin Author

cfriedline mannequin commented Aug 9, 2012

http://stackoverflow.com/questions/11884864/python-multiprocessing-creates-incorrect-pids

Jesse Noller suggested that I submit this stack overflow as a bug so it can be examined. As the problem is kind of domain specific, it's not easy to come up with a test case that can reproduce the issue.

Please let me know what more information I can provide.

Thanks!

@cfriedline cfriedline mannequin added the type-crash A hard crash of the interpreter, possibly with a core dump label Aug 9, 2012
@pitrou
Copy link
Member

pitrou commented Aug 9, 2012

Your report is rather ambiguous. Do you claim that said pids don't exist, or that they don't appear in "top"?

Also, do you manage to reproduce without R, or do you need to use R to reproduce? I would suspect an issue (or a misunderstanding on your part) with the R-Python bindings, rather than a bug in multiprocessing.

@cfriedline
Copy link
Mannequin Author

cfriedline mannequin commented Aug 9, 2012

That they don't appear in top. I was using that as a proxy for existence. I wrote a test case without R and realized (accidentally) that tracebacks will now flow up to the main process from the worker processes (which makes perfect sense, actually). I added explicit try/except to __get_valid_triplets. Will report back.

@pitrou
Copy link
Member

pitrou commented Aug 9, 2012

Chris added the comment:

That they don't appear in top. I was using that as a proxy for existence.

Well, I don't know in which way you use "top", but by default it will
only show you the N most CPU-consuming processes. If the R bindings, for
example (here begins the wild speculation on my part :-)), fork in the
background so as to run the R engine in a separate process, the
CPU-consuming processes would not be those directly created by
multiprocessing.

@cfriedline
Copy link
Mannequin Author

cfriedline mannequin commented Aug 9, 2012

Using top/'ps aux' is not the issue. I have code running now with the try/except. Will report back ASAP. I suspect that you're correct about something in the R pipeline failing. Hopefully, I'll get a good traceback.

@cfriedline
Copy link
Mannequin Author

cfriedline mannequin commented Aug 9, 2012

Also, when I said "...that tracebacks will now flow up to the main process from the worker processes (which makes perfect sense, actually)", I meant to say "...that tracebacks will NOT flow up to the main process from the worker processes (which makes perfect sense, actually)" ;-)

@cfriedline
Copy link
Mannequin Author

cfriedline mannequin commented Aug 9, 2012

Ugh, a server reboot seems to have cleared this all up (FML). It's been running well past the point of previous failure. I suppose I'll never know why it was failing. I'm going to close this out. Thanks for your time in helping me think this through.

@cfriedline cfriedline mannequin closed this as completed Aug 9, 2012
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@pitrou