As python gains more popularity, more and more applications run under
CPython in a common user desktop. The problem is that if a user has 2 or
more python apps running there is no way to identify them correctly from
the generic "Task/Process Manager Application" (in Windows/Linux/OSX/any
OS), because all appear as a "python" process. Take Ubuntu for an example. 

There are more cases when this is annoying:

- Imagine two python process (doing different things, of course) sending
messages to syslog: looks like all coming from "python" process.
- A bash script which wants to "killall" one application: how can it
identify a given python script between two or more "python" processes?

There are some methods [1][2] to work around it, but there are not just
Linux/BSD only, moreover, there are not included in the python standard
modules. 

[1] http://davyd.livejournal.com/166352.html
[2] http://code.google.com/p/procname/

I hope this issue gets considered in the community to fix it, I really
think this is important... :-)

Regards,
Marcelo

There was a similar request in Java bug database:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6299778

And I'd follow the same path: provide a way to build a launcher - 
a .exe file that simply starts python with the given script.

I suggest to close this issue, and join the discussion in issue4015

Uhm... ok, (looking at issue 4015) I agree with that method at some
point (tough is not exactly the same thing)... but only if the launcher
will not be for Windows only, and that solution would be consistent
across al platforms...

Just to know, there is no prctl equivalent in Windows?

Regards

If somebody would provide a patch that adds prctl to the posix module,
that would be fine with me - we have a long tradition of exposing all
available system calls if somebody wants them.

As for Windows: no, there is no equivalent of prctl. The task manager
often displays the Window title, or else the name of the executable module.

Fine, I'll try to make a patch based on the procname project in Google
Code. Here are some more links with more approaches to solve this issue
(while I learn how to make a patch :-) )

http://abock.org/2006/02/09/changing-process-name-in-mono/
http://shiny.thorne.id.au/2006/02/changing-python-process-titles.html
http://ubuntuforums.org/showthread.php?t=694331

Regards

This patch (to python 2.7 trunk) allows to call prctl() function from
Linux kernel to change the process name. It adds two methods to the os
module: os.getprocname() and os.setprocname().

Working example:

marcelo@marcelo-laptop:~/src/pytrunk$ ./python
Python 2.7a0 (trunk:71261M, Apr  5 2009, 16:32:55)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
[34831 refs]
>>> os.getprocname()
'./python'
[34833 refs]
>>> os.getpid()
5601
[34833 refs]
>>> os.setprocname('hello_process_name')
[34833 refs]
>>> os.getprocname()
'hello_process_name'
[34833 refs]

Before changing the process name:
marcelo@marcelo-laptop:~/src/pytrunk$ ps -fax | grep 5601
Warning: bad ps syntax, perhaps a bogus '-'? See
http://procps.sf.net/faq.html
 5601 pts/2    S+     0:00  |   \_ ./python
 5611 pts/3    S+     0:00      \_ grep 5601

After changing the process name:
marcelo@marcelo-laptop:~/src/pytrunk$ ps -fax | grep 5601
Warning: bad ps syntax, perhaps a bogus '-'? See
http://procps.sf.net/faq.html
 5601 pts/2    S+     0:00  |   \_ hello_process_name
 5635 pts/3    S+     0:00      \_ grep 5601

And "killall hello_process_name" works, Gnome Process monitor shows it
fine too. By now this is Linux only, but I hope to implement it also for
BSD (FreeBSD has it[1], OpenBSD [2] and NetBSD [3] too).

[1] http://fxr.watson.org/fxr/source/gen/setproctitle.c?v=FREEBSD-LIBC
[2]
http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/gen/setproctitle.c?rev=1.11;content-type=text%2Fx-cvsweb-markup
[3]
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/gen/setproctitle.c?rev=1.22&content-type=text/x-cvsweb-markup&only_with_tag=MAIN

Regards

As for os.getprocname() I think you can borrow some code from psutil
project [1] to implement it on FreeBSD, OS X and even Windows platforms.

[1] http://code.google.com/p/psutil

Some remarks about the patch:
1 - this line causes a buffer overrun:
    strncpy(argv[0], name , strlen(name));
A possible solution is to do like posix_putenv(): have a static PyString
that holds the memory, and just change the pointer argv[0]:
    argv[0] = PyString_AS_STRING(name);

2 - The function should update sys.argv as well. In this case,
os.getprocname is not necessary.

Great, it's my first patch to python and I'm very happy with your
comments. :-)

Thanks Amaury, do you think is better that I should take out
getprocname() if setprocname() changes sys.argv too?

When I got some spare time, I'll take a look at your suggestion and the
psutil project.

Thanks again

Please don't provide a wrapper around ptrctrl. Instead, expose prctl as
is (similar to how ioctl and fcntl get wrapped). It's ok to make some
convenience adjustments (e.g. allowing fewer parameters than the actual
call), and it's also ok to filter by option (for type safety), and be
lazy by not implementing all controls. However, renaming the system call
is not good, IMO.

> Please don't provide a wrapper around ptrctrl

prctl is not portable. I always thought that the premise of stdlib is to
provide portable interfaces.  BSD, for example, uses setprocname instead
of prctl.  Also, prctl does not modify the process name shown in "ps
uxww". Here's how PostgreSQL does this: http://tinyurl.com/mhjuqc.

Amaury Forgeot d'Arc, wrote:
  And I'd follow the same path: provide a way to build a launcher - 
  a .exe file that simply starts python with the given script.

Sounds good, but how would you expect to set the process name
for a subprocess, like a multiprocessing.Process? Make one shell script
for every process you want to launch and use exec? Maybe even dynamically 
create the scripts? This is not a solution.

> prctl is not portable. I always thought that the premise of stdlib is to
> provide portable interfaces.  BSD, for example, uses setprocname instead
> of prctl.  Also, prctl does not modify the process name shown in "ps
> uxww". Here's how PostgreSQL does this: http://tinyurl.com/mhjuqc.

Many parts of the stdlib are not portable.  It is perfectly appropriate
for the stdlib to expose a low-level, platform-specific API directly and
in a way which makes the Python API similarly low-level and non-portable.

That's not to say that a cross-platform API cannot *also* be provided,
though, assuming the functionality can be implemented *somehow* on other
platforms.  So I agree with Martin.  prctl should be exposed as-is.  If
someone would also like to expose similar functionality on on other
platforms, great.  And if someone would like to provide a cross-platform
API based on these, also great.

I wrote a wrapper around the PostgreSQL implementation of setproctitle 
(probably the most portable around). I've only tested it on Linux: 
probably will require tweaking constants to compile on other platforms 
(something postgres does at configure time) but the core functionality 
is surely well tested.

I've put the module on http://code.google.com/p/py-setproctitle/ : 
testing (yes, there is an unit test) and tweaking setup on other 
platforms is welcome.

Great, piro!

I'm taking a look at it, and it seems to use setproctitle() in BSD, and 
writes over the argv array "in most Sys-V like systems"; this includes 
Linux?

My question is because I think there's a better and supported method for 
Linux, that is, using prctl [1]. I read somewhere that changing argv 
causes some inconsistencies between programs who read /sys files, /proc 
files... or I don't remember what, but it is, in fact, not the 
*recommended* way. Prctl is. :-)

PR_SET_NAME and PR_GET_NAME parameters in prctl.h:
[1] http://www.kernel.org/doc/man-pages/online/pages/man2/prctl.2.html

Could this module be altered to use a prctl call in Linux (>2.6.9)?

Thanks a lot.

> I'm taking a look at it, and it seems to use setproctitle() in BSD, 
and 
> writes over the argv array "in most Sys-V like systems"; this 
includes 
> Linux?

Yes: Linux uses what in the source is referred as the 
PS_USE_CLOBBER_ARGV strategy: it writes over the area pointed by argv 
and by environ (after checked they are contiguous and moved away). 
Sounds scary, but I've tested that environ keeps working fine (the 
environ can be modified and forked processes receive the correct 
environment).

> My question is because I think there's a better and supported method 
for 
> Linux, that is, using prctl [1]. I read somewhere that changing argv 
> causes some inconsistencies between programs who read /sys files, 
/proc 
> files... or I don't remember what, but it is, in fact, not the 
> *recommended* way. Prctl is. :-)

This is interesting: do you have any reference?

I've tested with the difference between clobbering argv and call 
prctl: two demo programs are in the tools directory of the module 
project [2].

For what I observed, clobbering argv changes what shown in 
``/proc/PID/cmdline``, whereas prctl changes what can be read in 
``/proc/PID/status`` (and ``stat`` too). ``ps`` uses the former, but 
switches to the latter when calling ``ps a`` and ``ps f``. ``top`` 
toggles between the two pressing ``c``.

I don't know which method is better (well, I happen to prefer the 
extended visualization provided by the cmdline, which natively shows 
more detail than just the process name shown in ``status`` but this is 
probably subjective).

> Could this module be altered to use a prctl call in Linux (>2.6.9)?

I think is probably better to have both strings updated: I'd prefer 
this behavior instead of the title being set in different places on 
different Linux versions.

[2] http://code.google.com/p/py-setproctitle/source/browse/tools/

2009/12/6 Daniele Varrazzo <report@bugs.python.org>:
>> My question is because I think there's a better and supported method
> for
>> Linux, that is, using prctl [1]. I read somewhere that changing argv
>> causes some inconsistencies between programs who read /sys files,
> /proc
>> files... or I don't remember what, but it is, in fact, not the
>> *recommended* way. Prctl is. :-)
>
> This is interesting: do you have any reference?

For example, take a look at the comments here:
http://abock.org/2006/02/09/changing-process-name-in-mono

It seems that some utilities and programs (killall,
gnome-system-monitor, and so on) looks for the process name in
/proc/PID/status, not in /proc/PID/cmdline, so it should be better (in
Linux), to modify both, /proc/PID/cmdline (changing argv) *and*
/proc/PID/status (calling prctl()).

I installed py-setproctitle, and can't kill it with killall once I set
the name to "hello", for example. :-(

[1]+  Detenido                python
marcelo@marcelo-laptop:~/src/py-setproctitle$ ps a
  PID TTY      STAT   TIME COMMAND
 1030 tty4     Ss+    0:00 /sbin/getty -8 38400 tty4
 1033 tty5     Ss+    0:00 /sbin/getty -8 38400 tty5
 1049 tty2     Ss+    0:00 /sbin/getty -8 38400 tty2
 1050 tty3     Ss+    0:00 /sbin/getty -8 38400 tty3
 1052 tty6     Ss+    0:00 /sbin/getty -8 38400 tty6
 1349 tty7     Ss+   17:14 /usr/bin/X :0 -br -verbose -auth
/var/run/gdm/auth-for-gdm-XhETSO/database -nolisten tcp vt7
 2225 tty1     Ss+    0:00 /sbin/getty -8 38400 tty1
10344 pts/0    Ss+    0:00 /bin/bash -l
11923 pts/1    Ss     0:00 bash
12068 pts/2    Ss+    0:00 bash
12485 pts/1    T      0:00 hello
12496 pts/1    R+     0:00 ps a
marcelo@marcelo-laptop:~/src/py-setproctitle$ killall hello
hello: proceso no encontrado

Another example: The gnome-system-monitor still lists the 'hello'
process (PID 12485) like 'python'. So we should try

> I've tested with the difference between clobbering argv and call
> prctl: two demo programs are in the tools directory of the module
> project [2].
>
> For what I observed, clobbering argv changes what shown in
> ``/proc/PID/cmdline``, whereas prctl changes what can be read in
> ``/proc/PID/status`` (and ``stat`` too). ``ps`` uses the former, but
> switches to the latter when calling ``ps a`` and ``ps f``. ``top``
> toggles between the two pressing ``c``.

Sorry, but here (Ubuntu 9.10) it works the other way around: "ps"
lists the /proc/PID/status -> name field and "ps a" lists the
/proc/PID/cmdline. But I got the explanation. :-)

> I think is probably better to have both strings updated: I'd prefer
> this behavior instead of the title being set in different places on
> different Linux versions.

IMHO, I'd prefer both things: to change the argv array *and* calling
prctl() if possible (Linux >2.6.9). If Linux is lower than 2.6.9,
fallback to change argv only.

Regards
-- 
Marcelo F. Fernández
Buenos Aires, Argentina
Licenciado en Sistemas - CCNA

E-Mail: marcelo.fidel.fernandez@gmail.com
Blog: http://blog.marcelofernandez.info
Twitter: http://twitter.com/fidelfernandez

> It seems that some utilities and programs (killall,
> gnome-system-monitor, and so on) looks for the process name in
> /proc/PID/status, not in /proc/PID/cmdline, so it should be better
> (in Linux), to modify both, /proc/PID/cmdline (changing argv) *and*
> /proc/PID/status (calling prctl()).

> I installed py-setproctitle, and can't kill it with killall once I 
> set the name to "hello", for example. :-(

Just released setproctitle 0.2 where I also call prctl() if available. 
It is actually the string used by killall.

>> For what I observed, clobbering argv changes what shown in
>> ``/proc/PID/cmdline``, whereas prctl changes what can be read in
>> ``/proc/PID/status`` (and ``stat`` too). ``ps`` uses the former, 
but
>> switches to the latter when calling ``ps a`` and ``ps f``. ``top``
>> toggles between the two pressing ``c``.

> Sorry, but here (Ubuntu 9.10) it works the other way around: "ps"
> lists the /proc/PID/status -> name field and "ps a" lists the
> /proc/PID/cmdline. But I got the explanation. :-)

Yup, sorry: it was the other way round :P

2009/12/6 Daniele Varrazzo <report@bugs.python.org>:
>
> Just released setproctitle 0.2 where I also call prctl() if available.
> It is actually the string used by killall.

Great!

I've just tested it and it works fine here... Any possibility this
module can be included in the regular python standard library, in the
future?

Thanks!

> I've just tested it and it works fine here... Any possibility this
> module can be included in the regular python standard library, in the
> future?

Only in the far future. I don't think the Python standard library should
include a module whose version number is 0.2.

I propose to close this issue as "won't fix", with reference to the
external setproctitle module.

>> I've just tested it and it works fine here... Any possibility this
>> module can be included in the regular python standard library, in the
>> future?

>Only in the far future. I don't think the Python standard library 
> should include a module whose version number is 0.2.

I agree: I started the project 4-5 days ago and, albeit very limited in
its scope, for portability reasons is way more complex than a simple
syscall and needs to be tested on many other platforms.

2009/12/7 Daniele Varrazzo <report@bugs.python.org>:
>
> Daniele Varrazzo <piro@develer.com> added the comment:
>
>>> I've just tested it and it works fine here... Any possibility this
>>> module can be included in the regular python standard library, in the
>>> future?
>
>>Only in the far future. I don't think the Python standard library
>> should include a module whose version number is 0.2.
>
> I agree: I started the project 4-5 days ago and, albeit very limited in
> its scope, for portability reasons is way more complex than a simple
> syscall and needs to be tested on many other platforms.
>

Of course, when I said "future" I meant "when this project is stable
and tested enough"... :-)

Thanks

Hi,

just scanned the messages. As I read it "wontfix" actually means:

Ats some point in the future when the "setproctitle" project (http://code.google.com/p/py-setproctitle/source/list) has matured it will be reconsidered, correct?

Yes sorry for reopening this after quite some time (~6 months), just happened to pop up in my inbox due to the latest nosy list changes...

If it has matured, has shown to be as a "best of breed" library of it's type, and has gone through the PEP process, it could make it. That takes quite a bit of time and isn't likely to occur within the next few years (3.3 at the earliest assuming the project matures quickly).

setproctitle is quite stable, my company uses it in production environment very heavily with python 2.x. Probably its users base is not huge, but that's because it's a relatively specialized tool.

Python3 porting is not straightforward because python2 exports the original argv pointer using Py_GetArgcArgv() whereas python3 only exports a decoded version in a wchar_t* array. This may not be a showstopper: probably the original argv can still be found scanning backwards from environ: I want to test it but I haven't had requests for it yet, so it wasn't a top priority.

So, while I'd be pleased to have it included in the stdlib, I'm not really convinced it would be useful to such a large audience. Anyway I'm available for any improvement it would make the tool more useful and to anybody eager to push for its inclusion.

> So, while I'd be pleased to have it included in the stdlib, I'm not
> really convinced it would be useful to such a large audience.

I think different, if python becomes even more successful to be used in desktop apps, this feature will be *very* important, because the reasons I've exposed in this thread.

Regards

> Ats some point in the future when the "setproctitle" project
> (http://code.google.com/p/py-setproctitle/source/list) has matured it
> will be reconsidered, correct?

No. It will also be required that it's authors agree to contribute it to
Python, agree to stop maintaining the out-of-Python version
(eventually), and agree to fill out a contributor form.

> No. It will also be required that it's authors agree to contribute it to
> Python, agree to stop maintaining the out-of-Python version
> (eventually), and agree to fill out a contributor form.

I would have no problem with these points if the module is considered worthy for stdlib inclusion.

Hello everybody,

I've finished porting the module to Python 3, so I think it's a good starting point for considering its inclusion into the stdlib. The source is already out; tomorrow I'll test for regressions on mac osx and release the package.

I'm not going to champion a PEP for its inclusion though, as I'm not the right person to do what described in the pep approval process. I have no objection if somebody wanted to push for it anyway and to fulfill the requirements outlined by Martin.

Marcelo, I think it's up to you :)

I use the setproctitle module extensively. It has worked flawlessly.

What would be needed for this to get accepted? I realize one shouldn't stress such a decision, but still I feel this is something the standard library should be able to do.

My justification for inclusion into the standard library is that one has to make a workaround on systems where setproctitle ISN'T installed (I don't feel such a small component should be a requirement for production use.)

Everywhere I use setproctitle, I have to go:

    try:
        from setproctitle import setproctitle
    except ImportError:
        setproctitle = lambda t: None

Which is not only rather lengthy, but also IMO entirely unnecessary. As I noted, setting the process title is mostly for convenience and aesthetics, and therefore it's hard to justify the dependency. For that reason I think standard library inclusion is more relevant than otherwise.

So, are there any other reasons to wait with this other than the need of a PEP and proposing that to python-dev or whatever forum is best suited?

I'd rather not reopen this issue. It was too far-ranging, and has failed to get a specific solution. Please stop posting to this closed issue; if you want to contribute, please open a new one.

I think the inclusion of the module should see a discussion on python-dev first. It then also needs to be code-reviewed, which in turn needs a committer who either volunteers to do all this work, or who is willing to take the recommendation of some other reviewer.

My shallow review of the module is I would prefer to see the code structure revised. For example, c.h should go, spt_config.h should be integrated with autoconf, strlpcpy.c should go, spt_setup.c should go (IIUC) - IOW, this would need to be reformulated as a patch to the Python source tree. Of course, I can understand if Daniele would only start doing so if there was a chance that the functionality and approach is actually acceptable.

> If somebody would provide a patch that adds prctl to the posix module,
> that would be fine with me - we have a long tradition of exposing all
> available system calls if somebody wants them.

Just for the record, I was about to try to do this, when I realized that exposing prctl requires expecting a variable number of arguments with variable types.
It turns out providing a Python wrapper for such a kind of C API is just a pain.
There's an implementation though, handling 2 args only (instead of 5):
https://github.com/seveas/python-prctl/blob/master/_prctlmodule.c#L9
Considering that this also Linux-only, I don't think it really worths the effort.

> Just for the record, I was about to try to do this, when I realized that exposing prctl requires expecting a variable number of arguments with variable types.
> It turns out providing a Python wrapper for such a kind of C API is just a pain.
> There's an implementation though, handling 2 args only (instead of 5):
> https://github.com/seveas/python-prctl/blob/master/_prctlmodule.c#L9
> Considering that this also Linux-only, I don't think it really worths the effort.

It's actually simpler than you think: I don't think any of the existing
controls uses arg4 and arg5. PR_MCE_KILL uses arg3, though. prctl is
really a set of system calls, and should be wrapped as such: each
control needs to be supported specifically. However, several of them
are similar, i.e. take either no argument, or have

Whether it's worth supporting it, I don't know. I stand by my original
proposition: if somebody would contribute this, I'd be in favor of
including it. I'm unlikely to write it on my own from scratch, though.

There are actually a few implementations on pypi, just search for
prctl.  At least one of them is pretty decent IIRC but I can't
remember which one I looked at in detail before.  Anyway, they would
certainly be a reasonable starting point for python inclusion.

Isn't this "python script doesn't show up in top correctly" issue a matter of "#!/usr/bin/env python3"?

If you "#!/usr/bin/python3" (for example) instead, top seems happy.