classification
Title: python shouldn't need username in passwd database
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Python startup should not require passwd entry
View: 10496
Assigned To: Nosy List: aikinci, arekm, eric.araujo, georg.brandl, loewis, nadeem.vawda, vstinner
Priority: normal Keywords: buildbot, easy, patch

Created on 2012-03-09 09:21 by arekm, last changed 2012-03-13 00:07 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
issue14238.patch aikinci, 2012-03-12 21:44 review
Messages (12)
msg155215 - (view) Author: Arkadiusz Miskiewicz Arkadiusz Miskiewicz (arekm) Date: 2012-03-09 09:21
I'm trying to run python from UID which doesn't have entry in passwd database. This fails with:

 Traceback (most recent call last):
File "/usr/share/python2.7/site.py", line 567, in <module>
File "/usr/share/python2.7/site.py", line 549, in main
File "/usr/share/python2.7/site.py", line 278, in addusersitepackages
File "/usr/share/python2.7/site.py", line 253, in getusersitepackages
File "/usr/share/python2.7/site.py", line 243, in getuserbase
File "/usr/share/python2.7/sysconfig.py", line 522, in get_config_var
File "/usr/share/python2.7/sysconfig.py", line 426, in get_config_vars
File "/usr/share/python2.7/sysconfig.py", line 184, in _getuserbase
File "/usr/share/python2.7/sysconfig.py", line 171, in joinuser
File "/usr/share/python2.7/posixpath.py", line 260, in expanduser
KeyError: 'getpwuid(): uid not found: 51'
msg155216 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2012-03-09 09:31
This looks similar to a sporadic failure on the debian bigmem buildbot:

http://www.python.org/dev/buildbot/all/builders/AMD64%20debian%20bigmem%203.x/builds/168/steps/test/logs/stdio

    ERROR: test_expanduser (test.test_posixpath.PosixPathTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/var/tmpfs/martin.vonloewis/3.x.loewis-parallel2/build/Lib/test/test_posixpath.py", line 305, in test_expanduser
        home = pwd.getpwuid(os.getuid()).pw_dir
    KeyError: 'getpwuid(): uid not found: 5025'
msg155218 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-03-09 09:37
I agree -- there should be a guard here that just doesn't add user site directories if the lookup fails.
msg155279 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-10 02:14
> there should be a guard here that just doesn't add user site directories if the lookup fails.
Agreed too.  site is easy to fix (I can do it or review a patch), but we also need to fix sysconfig; I think the right thing to do would be not to have a config var named 'userbase' when run from a UID without passwd entry.  This implies that the functions listed under http://docs.python.org/library/sysconfig#installation-paths would also need to omit *user schemes.
msg155465 - (view) Author: Ali Ikinci (aikinci) Date: 2012-03-12 19:38
I have started to work on this.
msg155486 - (view) Author: Ali Ikinci (aikinci) Date: 2012-03-12 21:44
This patch fixes the issue but needs review. It is my first patch ever so be nice :)
msg155488 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-12 21:50
Thanks you for your contribution!  I hope you’ll find fun bugs to make patches for.

Your patch looks too simplistic; in the discussion, we agreed that if the lookup fails, the site module should not add user site directories (see http://docs.python.org/library/site#site.USER_SITE or PEP 370), and your patch does not seem to do that.

Moreover, this bug has repercussions on the sysconfig module (see my previous message), which needs edits too.

BTW, I don’t know how we can test a patch for this bug; arekm, how do you run Python from a UID without passwd entry?
msg155493 - (view) Author: Arkadiusz Miskiewicz Arkadiusz Miskiewicz (arekm) Date: 2012-03-12 22:04
$  LC_ALL=C sudo -u '#9999' -g '#9999' /bin/sh -c 'unset HOME ; python -c ""'
Traceback (most recent call last):
  File "/usr/share/python2.7/site.py", line 567, in <module>
    main()
  File "/usr/share/python2.7/site.py", line 549, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/share/python2.7/site.py", line 278, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/share/python2.7/site.py", line 253, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/share/python2.7/site.py", line 243, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/usr/share/python2.7/sysconfig.py", line 522, in get_config_var
    return get_config_vars().get(name)
  File "/usr/share/python2.7/sysconfig.py", line 426, in get_config_vars
    _CONFIG_VARS['userbase'] = _getuserbase()
  File "/usr/share/python2.7/sysconfig.py", line 184, in _getuserbase
    return env_base if env_base else joinuser("~", ".local")
  File "/usr/share/python2.7/sysconfig.py", line 171, in joinuser
    return os.path.expanduser(os.path.join(*args))
  File "/usr/share/python2.7/posixpath.py", line 260, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 9999'
msg155494 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-03-12 22:07
See also the issue #10496.
msg155499 - (view) Author: Ali Ikinci (aikinci) Date: 2012-03-12 22:18
I would suggest that this issue is the same as http://bugs.python.org/issue10496 like Victor pointed out and would say close it because it is a duplicate.
msg155529 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-13 00:04
Indeed.
msg155531 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-03-13 00:07
> This patch fixes the issue but needs review.
> It is my first patch ever so be nice :)

Python should not decide what is the default home directory. The issue is in the site module: I proposed nonexistent_user.patch in the issue #10496 to fix the issue in the site module instead.
History
Date User Action Args
2012-03-13 00:07:40vstinnersetmessages: + msg155531
2012-03-13 00:04:10eric.araujosetstatus: open -> closed
resolution: duplicate
messages: + msg155529

superseder: Python startup should not require passwd entry
stage: needs patch -> resolved
2012-03-12 22:18:44aikincisetmessages: + msg155499
2012-03-12 22:07:27vstinnersetnosy: + vstinner
messages: + msg155494
2012-03-12 22:04:56arekmsetmessages: + msg155493
2012-03-12 21:50:50eric.araujosetmessages: + msg155488
2012-03-12 21:44:32aikincisetfiles: + issue14238.patch
keywords: + patch
messages: + msg155486
2012-03-12 19:38:28aikincisetnosy: + aikinci
messages: + msg155465
2012-03-10 02:14:07eric.araujosetmessages: + msg155279
2012-03-09 16:41:56pitrousetkeywords: + easy
nosy: + eric.araujo

versions: + Python 3.2
2012-03-09 09:37:01georg.brandlsetnosy: + georg.brandl
messages: + msg155218
2012-03-09 09:31:56nadeem.vawdasettype: behavior
components: + Library (Lib), - Interpreter Core
versions: + Python 3.3
keywords: + buildbot
nosy: + loewis, nadeem.vawda

messages: + msg155216
stage: needs patch
2012-03-09 09:21:59arekmcreate